2024-09-13T08:53:26.059 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T08:53:26.076 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T08:53:26.167 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901010' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: true rbd cache policy: writeback global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 248 sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 sleep_before_teardown: 0 subset: 116/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN69Fh17vuaP8Jp0Ac+y2fLkte/crXDtelNrBYCTB0GJddGdwfnpR8+P0RNLu56vIql0leV37IuCRYMZCOomtCY= smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvUXqVvFJmICKsPIqTyb3eS2g2iw1eavX67ABEGJpBOc85TRch+6fj5gQAV9489B8H96uTo8+SOys1rn0yKFBc= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T08:53:26.167 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa; will attempt to use it 2024-09-13T08:53:26.168 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa/tasks 2024-09-13T08:53:26.168 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T08:53:26.170 INFO:teuthology.task.internal:Checking packages... 2024-09-13T08:53:26.190 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '48a7de7e6e1701a88fd571c7e34908ac4b8ed412' 2024-09-13T08:53:26.190 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T08:53:26.190 INFO:teuthology.packaging:ref: None 2024-09-13T08:53:26.190 INFO:teuthology.packaging:tag: None 2024-09-13T08:53:26.190 INFO:teuthology.packaging:branch: reef 2024-09-13T08:53:26.190 INFO:teuthology.packaging:sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T08:53:26.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=reef 2024-09-13T08:53:26.424 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-843-g2c01458a-1jammy 2024-09-13T08:53:26.425 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T08:53:26.432 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T08:53:26.432 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T08:53:26.448 INFO:teuthology.task.internal:Saving configuration 2024-09-13T08:53:26.460 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T08:53:26.467 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T08:53:26.498 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:48:08.242306', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBsIJMmK3smWpanJ6gliZnYzlwpDKoLKTFo4tGW8Lgmd'} 2024-09-13T08:53:26.524 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 08:48:08.241149', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINx9UcPIsCue83c60RqsxfoV5xZwbnQ3LROEyyU97tQo'} 2024-09-13T08:53:26.524 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T08:53:26.534 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-13T08:53:26.534 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-13T08:53:26.534 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T08:53:26.590 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa0b9e6b640>, signals=[15]) 2024-09-13T08:53:26.590 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T08:53:26.596 INFO:teuthology.task.internal:Opening connections... 2024-09-13T08:53:26.596 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T08:53:26.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:53:26.679 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T08:53:26.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:53:26.762 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T08:53:26.769 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-13T08:53:26.774 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-13T08:53:26.774 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Ubuntu" 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="22.04" 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_CODENAME=jammy 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:ID=ubuntu 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE=debian 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T08:53:26.824 INFO:teuthology.orchestra.run.smithi031.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T08:53:26.825 INFO:teuthology.orchestra.run.smithi031.stdout:UBUNTU_CODENAME=jammy 2024-09-13T08:53:26.825 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-13T08:53:26.843 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-09-13T08:53:26.848 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-09-13T08:53:26.848 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-09-13T08:53:26.899 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T08:53:26.899 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="Ubuntu" 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="22.04" 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_CODENAME=jammy 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:ID=ubuntu 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE=debian 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T08:53:26.900 INFO:teuthology.orchestra.run.smithi060.stdout:UBUNTU_CODENAME=jammy 2024-09-13T08:53:26.900 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-09-13T08:53:26.918 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T08:53:26.926 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T08:53:26.981 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T08:53:26.982 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-13T08:53:26.984 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-09-13T08:53:26.989 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T08:53:26.996 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T08:53:26.996 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-13T08:53:27.033 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-09-13T08:53:27.048 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T08:53:27.099 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T08:53:27.113 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T08:53:27.113 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T08:53:27.114 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T08:53:27.114 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T08:53:27.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:53:27.118 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-13T08:53:27.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T08:53:27.119 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-13T08:53:27.164 INFO:teuthology.orchestra.run.smithi031.stdout:5.15.0-56-generic 2024-09-13T08:53:27.164 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.15.0-56-generic 2024-09-13T08:53:27.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-get clean 2024-09-13T08:53:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:5.15.0-56-generic 2024-09-13T08:53:27.168 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.15.0-56-generic 2024-09-13T08:53:27.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get clean 2024-09-13T08:53:27.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get update 2024-09-13T08:53:27.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-get update 2024-09-13T08:53:27.421 INFO:teuthology.orchestra.run.smithi060.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T08:53:27.479 INFO:teuthology.orchestra.run.smithi031.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T08:53:27.537 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T08:53:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T08:53:27.619 INFO:teuthology.orchestra.run.smithi060.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T08:53:27.629 INFO:teuthology.orchestra.run.smithi060.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T08:53:27.671 INFO:teuthology.orchestra.run.smithi031.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T08:53:27.703 INFO:teuthology.orchestra.run.smithi031.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T08:53:27.729 INFO:teuthology.orchestra.run.smithi060.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T08:53:27.734 INFO:teuthology.orchestra.run.smithi060.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T08:53:27.737 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T08:53:27.737 INFO:teuthology.orchestra.run.smithi060.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T08:53:27.758 INFO:teuthology.orchestra.run.smithi060.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T08:53:27.758 INFO:teuthology.orchestra.run.smithi060.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T08:53:27.762 INFO:teuthology.orchestra.run.smithi060.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T08:53:27.762 INFO:teuthology.orchestra.run.smithi060.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T08:53:27.771 INFO:teuthology.orchestra.run.smithi060.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T08:53:27.773 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T08:53:27.777 INFO:teuthology.orchestra.run.smithi060.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T08:53:27.779 INFO:teuthology.orchestra.run.smithi060.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T08:53:27.779 INFO:teuthology.orchestra.run.smithi060.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T08:53:27.779 INFO:teuthology.orchestra.run.smithi060.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T08:53:27.789 INFO:teuthology.orchestra.run.smithi060.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T08:53:27.789 INFO:teuthology.orchestra.run.smithi060.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T08:53:27.810 INFO:teuthology.orchestra.run.smithi031.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T08:53:27.813 INFO:teuthology.orchestra.run.smithi031.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T08:53:27.813 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T08:53:27.849 INFO:teuthology.orchestra.run.smithi031.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T08:53:27.849 INFO:teuthology.orchestra.run.smithi031.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T08:53:27.853 INFO:teuthology.orchestra.run.smithi031.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T08:53:27.853 INFO:teuthology.orchestra.run.smithi031.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T08:53:27.858 INFO:teuthology.orchestra.run.smithi031.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T08:53:27.864 INFO:teuthology.orchestra.run.smithi031.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T08:53:27.865 INFO:teuthology.orchestra.run.smithi031.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T08:53:27.866 INFO:teuthology.orchestra.run.smithi031.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T08:53:27.866 INFO:teuthology.orchestra.run.smithi031.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T08:53:27.897 INFO:teuthology.orchestra.run.smithi031.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T08:53:27.897 INFO:teuthology.orchestra.run.smithi031.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T08:53:27.975 INFO:teuthology.orchestra.run.smithi060.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T08:53:28.027 INFO:teuthology.orchestra.run.smithi031.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T08:53:28.115 INFO:teuthology.orchestra.run.smithi060.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T08:53:28.165 INFO:teuthology.orchestra.run.smithi031.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T08:53:28.433 INFO:teuthology.orchestra.run.smithi060.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T08:53:28.468 INFO:teuthology.orchestra.run.smithi060.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T08:53:28.482 INFO:teuthology.orchestra.run.smithi060.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T08:53:28.483 INFO:teuthology.orchestra.run.smithi031.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T08:53:28.483 INFO:teuthology.orchestra.run.smithi060.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T08:53:28.518 INFO:teuthology.orchestra.run.smithi031.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T08:53:28.532 INFO:teuthology.orchestra.run.smithi031.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T08:53:28.533 INFO:teuthology.orchestra.run.smithi031.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T08:53:28.534 INFO:teuthology.orchestra.run.smithi031.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T08:53:28.552 INFO:teuthology.orchestra.run.smithi060.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T08:53:28.554 INFO:teuthology.orchestra.run.smithi060.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T08:53:28.569 INFO:teuthology.orchestra.run.smithi060.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T08:53:28.570 INFO:teuthology.orchestra.run.smithi060.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T08:53:28.601 INFO:teuthology.orchestra.run.smithi060.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T08:53:28.603 INFO:teuthology.orchestra.run.smithi031.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T08:53:28.618 INFO:teuthology.orchestra.run.smithi031.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T08:53:28.618 INFO:teuthology.orchestra.run.smithi031.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T08:53:28.620 INFO:teuthology.orchestra.run.smithi060.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T08:53:28.626 INFO:teuthology.orchestra.run.smithi060.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T08:53:28.626 INFO:teuthology.orchestra.run.smithi060.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T08:53:28.627 INFO:teuthology.orchestra.run.smithi060.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T08:53:28.653 INFO:teuthology.orchestra.run.smithi031.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T08:53:28.670 INFO:teuthology.orchestra.run.smithi031.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T08:53:28.676 INFO:teuthology.orchestra.run.smithi031.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T08:53:28.677 INFO:teuthology.orchestra.run.smithi031.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T08:53:28.678 INFO:teuthology.orchestra.run.smithi031.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T08:53:28.679 INFO:teuthology.orchestra.run.smithi031.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T08:53:28.679 INFO:teuthology.orchestra.run.smithi060.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T08:53:28.680 INFO:teuthology.orchestra.run.smithi060.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T08:53:28.680 INFO:teuthology.orchestra.run.smithi060.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T08:53:28.682 INFO:teuthology.orchestra.run.smithi060.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T08:53:28.710 INFO:teuthology.orchestra.run.smithi060.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T08:53:28.711 INFO:teuthology.orchestra.run.smithi060.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T08:53:28.711 INFO:teuthology.orchestra.run.smithi060.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T08:53:28.712 INFO:teuthology.orchestra.run.smithi060.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T08:53:28.713 INFO:teuthology.orchestra.run.smithi060.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T08:53:28.728 INFO:teuthology.orchestra.run.smithi031.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T08:53:28.728 INFO:teuthology.orchestra.run.smithi031.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T08:53:28.730 INFO:teuthology.orchestra.run.smithi031.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T08:53:28.761 INFO:teuthology.orchestra.run.smithi060.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T08:53:28.761 INFO:teuthology.orchestra.run.smithi031.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T08:53:28.761 INFO:teuthology.orchestra.run.smithi031.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T08:53:28.762 INFO:teuthology.orchestra.run.smithi031.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T08:53:28.763 INFO:teuthology.orchestra.run.smithi031.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T08:53:28.763 INFO:teuthology.orchestra.run.smithi031.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T08:53:28.809 INFO:teuthology.orchestra.run.smithi031.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T08:53:34.442 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 16.0 MB in 2s (7,122 kB/s) 2024-09-13T08:53:34.695 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 16.0 MB in 2s (7,254 kB/s) 2024-09-13T08:53:35.899 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-09-13T08:53:35.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T08:53:35.973 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-09-13T08:53:36.164 INFO:teuthology.orchestra.run.smithi060.stdout:Building dependency tree... 2024-09-13T08:53:36.165 INFO:teuthology.orchestra.run.smithi060.stdout:Reading state information... 2024-09-13T08:53:36.212 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-13T08:53:36.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T08:53:36.297 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-13T08:53:36.340 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: python2.7-minimal 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout:The following additional packages will be installed: 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T08:53:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T08:53:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T08:53:36.342 INFO:teuthology.orchestra.run.smithi060.stdout:Suggested packages: 2024-09-13T08:53:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T08:53:36.342 INFO:teuthology.orchestra.run.smithi060.stdout:Recommended packages: 2024-09-13T08:53:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: thermald 2024-09-13T08:53:36.396 INFO:teuthology.orchestra.run.smithi060.stdout:The following NEW packages will be installed: 2024-09-13T08:53:36.396 INFO:teuthology.orchestra.run.smithi060.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T08:53:36.396 INFO:teuthology.orchestra.run.smithi060.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T08:53:36.397 INFO:teuthology.orchestra.run.smithi060.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T08:53:36.398 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages will be upgraded: 2024-09-13T08:53:36.398 INFO:teuthology.orchestra.run.smithi060.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T08:53:36.491 INFO:teuthology.orchestra.run.smithi031.stdout:Building dependency tree... 2024-09-13T08:53:36.492 INFO:teuthology.orchestra.run.smithi031.stdout:Reading state information... 2024-09-13T08:53:36.579 INFO:teuthology.orchestra.run.smithi060.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T08:53:36.579 INFO:teuthology.orchestra.run.smithi060.stdout:Need to get 113 MB of archives. 2024-09-13T08:53:36.579 INFO:teuthology.orchestra.run.smithi060.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T08:53:36.580 INFO:teuthology.orchestra.run.smithi060.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T08:53:36.663 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T08:53:36.663 INFO:teuthology.orchestra.run.smithi031.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T08:53:36.663 INFO:teuthology.orchestra.run.smithi031.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T08:53:36.664 INFO:teuthology.orchestra.run.smithi031.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T08:53:36.664 INFO:teuthology.orchestra.run.smithi031.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T08:53:36.664 INFO:teuthology.orchestra.run.smithi031.stdout: python2.7-minimal 2024-09-13T08:53:36.664 INFO:teuthology.orchestra.run.smithi031.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T08:53:36.665 INFO:teuthology.orchestra.run.smithi031.stdout:The following additional packages will be installed: 2024-09-13T08:53:36.665 INFO:teuthology.orchestra.run.smithi031.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T08:53:36.665 INFO:teuthology.orchestra.run.smithi031.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T08:53:36.666 INFO:teuthology.orchestra.run.smithi031.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T08:53:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:Suggested packages: 2024-09-13T08:53:36.667 INFO:teuthology.orchestra.run.smithi031.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T08:53:36.667 INFO:teuthology.orchestra.run.smithi031.stdout:Recommended packages: 2024-09-13T08:53:36.667 INFO:teuthology.orchestra.run.smithi031.stdout: thermald 2024-09-13T08:53:36.719 INFO:teuthology.orchestra.run.smithi031.stdout:The following NEW packages will be installed: 2024-09-13T08:53:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T08:53:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T08:53:36.719 INFO:teuthology.orchestra.run.smithi031.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T08:53:36.721 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages will be upgraded: 2024-09-13T08:53:36.721 INFO:teuthology.orchestra.run.smithi031.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T08:53:36.774 INFO:teuthology.orchestra.run.smithi031.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T08:53:36.774 INFO:teuthology.orchestra.run.smithi031.stdout:Need to get 113 MB of archives. 2024-09-13T08:53:36.774 INFO:teuthology.orchestra.run.smithi031.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T08:53:36.774 INFO:teuthology.orchestra.run.smithi031.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T08:53:37.222 INFO:teuthology.orchestra.run.smithi031.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T08:53:37.320 INFO:teuthology.orchestra.run.smithi031.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T08:53:37.942 INFO:teuthology.orchestra.run.smithi060.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T08:53:38.214 INFO:teuthology.orchestra.run.smithi031.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T08:53:38.262 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T08:53:38.267 INFO:teuthology.orchestra.run.smithi031.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T08:53:38.347 INFO:teuthology.orchestra.run.smithi060.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T08:53:38.424 INFO:teuthology.orchestra.run.smithi031.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T08:53:38.447 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T08:53:38.946 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 113 MB in 2s (65.9 MB/s) 2024-09-13T08:53:39.138 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T08:53:40.525 INFO:teuthology.orchestra.run.smithi060.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T08:53:40.525 INFO:teuthology.orchestra.run.smithi060.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T08:53:40.525 INFO:teuthology.orchestra.run.smithi060.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T08:53:40.860 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T08:53:40.941 INFO:teuthology.orchestra.run.smithi060.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T08:53:41.185 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T08:53:41.189 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T08:53:41.363 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 113 MB in 5s (25.0 MB/s) 2024-09-13T08:53:41.501 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:53:41.535 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T08:53:43.549 INFO:teuthology.orchestra.run.smithi060.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T08:53:43.553 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T08:53:43.615 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:53:43.877 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T08:53:43.898 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T08:53:43.976 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:53:44.530 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T08:53:44.551 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T08:53:44.588 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:53:45.901 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T08:53:45.922 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T08:53:46.002 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:53:46.454 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T08:53:46.474 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T08:53:46.511 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:53:51.613 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T08:53:51.737 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T08:53:52.121 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T08:53:52.241 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T08:53:52.392 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T08:53:52.516 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T08:53:52.569 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T08:53:52.591 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T08:53:52.635 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T08:53:52.957 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T08:53:53.086 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T08:53:53.381 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T08:53:53.403 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T08:53:53.440 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T08:53:59.220 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T08:53:59.257 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T08:53:59.295 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:53:59.384 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T08:53:59.422 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T08:53:59.459 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:01.942 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T08:54:02.064 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T08:54:02.356 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T08:54:02.494 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:02.596 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T08:54:02.620 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T08:54:02.729 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T08:54:02.748 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:03.885 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T08:54:04.043 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:04.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T08:54:04.248 INFO:teuthology.orchestra.run.smithi060.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:54:04.248 INFO:teuthology.orchestra.run.smithi060.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T08:54:04.249 INFO:teuthology.orchestra.run.smithi060.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T08:54:04.249 INFO:teuthology.orchestra.run.smithi060.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T08:54:04.314 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:04.411 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:05.677 INFO:teuthology.orchestra.run.smithi031.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:54:05.677 INFO:teuthology.orchestra.run.smithi031.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T08:54:05.677 INFO:teuthology.orchestra.run.smithi031.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T08:54:05.677 INFO:teuthology.orchestra.run.smithi031.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T08:54:05.859 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:05.879 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:07.280 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T08:54:07.293 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:07.406 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T08:54:07.532 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:07.592 INFO:teuthology.orchestra.run.smithi060.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T08:54:07.592 INFO:teuthology.orchestra.run.smithi060.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T08:54:08.728 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T08:54:08.929 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T08:54:09.889 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T08:54:09.949 INFO:teuthology.orchestra.run.smithi031.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T08:54:09.949 INFO:teuthology.orchestra.run.smithi031.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T08:54:22.201 INFO:teuthology.orchestra.run.smithi060.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T08:54:22.201 INFO:teuthology.orchestra.run.smithi060.stdout:Sourcing file `/etc/default/grub' 2024-09-13T08:54:22.210 INFO:teuthology.orchestra.run.smithi060.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T08:54:22.228 INFO:teuthology.orchestra.run.smithi060.stdout:Generating grub configuration file ... 2024-09-13T08:54:22.505 INFO:teuthology.orchestra.run.smithi060.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T08:54:22.529 INFO:teuthology.orchestra.run.smithi060.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T08:54:22.928 INFO:teuthology.orchestra.run.smithi060.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:54:22.933 INFO:teuthology.orchestra.run.smithi060.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T08:54:23.007 INFO:teuthology.orchestra.run.smithi060.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T08:54:23.013 INFO:teuthology.orchestra.run.smithi060.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T08:54:23.161 INFO:teuthology.orchestra.run.smithi060.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T08:54:23.161 INFO:teuthology.orchestra.run.smithi060.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T08:54:23.161 INFO:teuthology.orchestra.run.smithi060.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T08:54:23.210 INFO:teuthology.orchestra.run.smithi060.stdout:done 2024-09-13T08:54:23.722 DEBUG:teuthology.orchestra.run.smithi060:> dpkg -s linux-image-generic 2024-09-13T08:54:23.742 INFO:teuthology.orchestra.run.smithi060.stdout:Package: linux-image-generic 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Status: install ok installed 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Priority: optional 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Section: kernel 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Installed-Size: 21 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: amd64 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Source: linux-meta 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Version: 5.15.0.121.121 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Recommends: thermald 2024-09-13T08:54:23.743 INFO:teuthology.orchestra.run.smithi060.stdout:Description: Generic Linux kernel image 2024-09-13T08:54:23.744 INFO:teuthology.orchestra.run.smithi060.stdout: This package will always depend on the latest generic kernel image 2024-09-13T08:54:23.744 INFO:teuthology.orchestra.run.smithi060.stdout: available. 2024-09-13T08:54:23.745 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T08:54:23.745 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T08:54:23.745 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T08:54:23.745 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T08:54:23.745 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T08:54:23.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get clean 2024-09-13T08:54:23.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get update 2024-09-13T08:54:24.069 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T08:54:24.078 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T08:54:24.096 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T08:54:24.190 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T08:54:25.366 INFO:teuthology.orchestra.run.smithi031.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T08:54:25.366 INFO:teuthology.orchestra.run.smithi031.stdout:Sourcing file `/etc/default/grub' 2024-09-13T08:54:25.379 INFO:teuthology.orchestra.run.smithi031.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T08:54:25.398 INFO:teuthology.orchestra.run.smithi031.stdout:Generating grub configuration file ... 2024-09-13T08:54:25.588 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-09-13T08:54:25.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T08:54:25.662 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-09-13T08:54:25.750 INFO:teuthology.orchestra.run.smithi031.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T08:54:25.777 INFO:teuthology.orchestra.run.smithi031.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T08:54:25.851 INFO:teuthology.orchestra.run.smithi060.stdout:Building dependency tree... 2024-09-13T08:54:25.852 INFO:teuthology.orchestra.run.smithi060.stdout:Reading state information... 2024-09-13T08:54:26.031 INFO:teuthology.orchestra.run.smithi060.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T08:54:26.031 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T08:54:26.031 INFO:teuthology.orchestra.run.smithi060.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T08:54:26.031 INFO:teuthology.orchestra.run.smithi060.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T08:54:26.031 INFO:teuthology.orchestra.run.smithi060.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T08:54:26.032 INFO:teuthology.orchestra.run.smithi060.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T08:54:26.032 INFO:teuthology.orchestra.run.smithi060.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T08:54:26.032 INFO:teuthology.orchestra.run.smithi060.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T08:54:26.073 INFO:teuthology.orchestra.run.smithi060.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T08:54:26.075 DEBUG:teuthology.orchestra.run.smithi060:> dpkg -s linux-image-generic 2024-09-13T08:54:26.134 INFO:teuthology.orchestra.run.smithi060.stdout:Package: linux-image-generic 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Status: install ok installed 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Priority: optional 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Section: kernel 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Installed-Size: 21 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: amd64 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Source: linux-meta 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Version: 5.15.0.121.121 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T08:54:26.135 INFO:teuthology.orchestra.run.smithi060.stdout:Recommends: thermald 2024-09-13T08:54:26.136 INFO:teuthology.orchestra.run.smithi060.stdout:Description: Generic Linux kernel image 2024-09-13T08:54:26.136 INFO:teuthology.orchestra.run.smithi060.stdout: This package will always depend on the latest generic kernel image 2024-09-13T08:54:26.136 INFO:teuthology.orchestra.run.smithi060.stdout: available. 2024-09-13T08:54:26.137 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-09-13T08:54:26.140 INFO:teuthology.orchestra.run.smithi031.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:54:26.146 INFO:teuthology.orchestra.run.smithi031.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T08:54:26.183 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.c17nHq6TgX 2024-09-13T08:54:26.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub/grub.cfg /tmp/tmp.c17nHq6TgX 2024-09-13T08:54:26.217 INFO:teuthology.orchestra.run.smithi031.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T08:54:26.222 INFO:teuthology.orchestra.run.smithi031.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T08:54:26.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.c17nHq6TgX 2024-09-13T08:54:26.357 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.c17nHq6TgX is 10KB 2024-09-13T08:54:26.370 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.c17nHq6TgX 2024-09-13T08:54:26.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T08:54:26.397 INFO:teuthology.orchestra.run.smithi031.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T08:54:26.398 INFO:teuthology.orchestra.run.smithi031.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T08:54:26.398 INFO:teuthology.orchestra.run.smithi031.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T08:54:26.432 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T08:54:26.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T08:54:26.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T08:54:26.446 INFO:teuthology.orchestra.run.smithi031.stdout:done 2024-09-13T08:54:26.501 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T08:54:26.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo update-grub 2024-09-13T08:54:26.975 DEBUG:teuthology.orchestra.run.smithi031:> dpkg -s linux-image-generic 2024-09-13T08:54:26.997 INFO:teuthology.orchestra.run.smithi031.stdout:Package: linux-image-generic 2024-09-13T08:54:26.997 INFO:teuthology.orchestra.run.smithi031.stdout:Status: install ok installed 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Priority: optional 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Section: kernel 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Installed-Size: 21 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: amd64 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Source: linux-meta 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Version: 5.15.0.121.121 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Recommends: thermald 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout:Description: Generic Linux kernel image 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout: This package will always depend on the latest generic kernel image 2024-09-13T08:54:26.998 INFO:teuthology.orchestra.run.smithi031.stdout: available. 2024-09-13T08:54:26.999 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T08:54:26.999 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T08:54:26.999 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T08:54:26.999 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T08:54:26.999 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T08:54:26.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-get clean 2024-09-13T08:54:27.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-get update 2024-09-13T08:54:27.293 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T08:54:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T08:54:27.497 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T08:54:27.574 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T08:54:27.682 INFO:teuthology.orchestra.run.smithi060.stderr:Sourcing file `/etc/default/grub' 2024-09-13T08:54:27.684 INFO:teuthology.orchestra.run.smithi060.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T08:54:27.691 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-09-13T08:54:27.879 INFO:teuthology.orchestra.run.smithi060.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T08:54:27.886 INFO:teuthology.orchestra.run.smithi060.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T08:54:28.118 INFO:teuthology.orchestra.run.smithi060.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:54:28.123 INFO:teuthology.orchestra.run.smithi060.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T08:54:28.167 INFO:teuthology.orchestra.run.smithi060.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T08:54:28.173 INFO:teuthology.orchestra.run.smithi060.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T08:54:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T08:54:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T08:54:28.338 INFO:teuthology.orchestra.run.smithi060.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T08:54:28.349 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-09-13T08:54:28.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-09-13T08:54:28.773 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-13T08:54:28.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T08:54:28.847 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-13T08:54:29.038 INFO:teuthology.orchestra.run.smithi031.stdout:Building dependency tree... 2024-09-13T08:54:29.038 INFO:teuthology.orchestra.run.smithi031.stdout:Reading state information... 2024-09-13T08:54:29.213 INFO:teuthology.orchestra.run.smithi031.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T08:54:29.213 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T08:54:29.213 INFO:teuthology.orchestra.run.smithi031.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T08:54:29.213 INFO:teuthology.orchestra.run.smithi031.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T08:54:29.214 INFO:teuthology.orchestra.run.smithi031.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T08:54:29.214 INFO:teuthology.orchestra.run.smithi031.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T08:54:29.214 INFO:teuthology.orchestra.run.smithi031.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T08:54:29.214 INFO:teuthology.orchestra.run.smithi031.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T08:54:29.255 INFO:teuthology.orchestra.run.smithi031.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T08:54:29.256 DEBUG:teuthology.orchestra.run.smithi031:> dpkg -s linux-image-generic 2024-09-13T08:54:29.273 INFO:teuthology.orchestra.run.smithi031.stdout:Package: linux-image-generic 2024-09-13T08:54:29.273 INFO:teuthology.orchestra.run.smithi031.stdout:Status: install ok installed 2024-09-13T08:54:29.273 INFO:teuthology.orchestra.run.smithi031.stdout:Priority: optional 2024-09-13T08:54:29.273 INFO:teuthology.orchestra.run.smithi031.stdout:Section: kernel 2024-09-13T08:54:29.273 INFO:teuthology.orchestra.run.smithi031.stdout:Installed-Size: 21 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: amd64 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:Source: linux-meta 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:Version: 5.15.0.121.121 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:Recommends: thermald 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout:Description: Generic Linux kernel image 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout: This package will always depend on the latest generic kernel image 2024-09-13T08:54:29.274 INFO:teuthology.orchestra.run.smithi031.stdout: available. 2024-09-13T08:54:29.275 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-13T08:54:29.319 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.rriLwaJiR7 2024-09-13T08:54:29.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub/grub.cfg /tmp/tmp.rriLwaJiR7 2024-09-13T08:54:29.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.rriLwaJiR7 2024-09-13T08:54:29.487 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.rriLwaJiR7 is 10KB 2024-09-13T08:54:29.500 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.rriLwaJiR7 2024-09-13T08:54:29.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T08:54:29.566 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T08:54:29.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T08:54:29.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T08:54:29.660 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T08:54:29.660 DEBUG:teuthology.orchestra.run.smithi031:> sudo update-grub 2024-09-13T08:54:31.200 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub' 2024-09-13T08:54:31.217 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T08:54:31.225 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-13T08:54:31.463 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T08:54:31.470 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T08:54:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T08:54:31.719 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T08:54:31.764 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T08:54:31.769 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T08:54:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T08:54:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T08:54:31.885 INFO:teuthology.orchestra.run.smithi031.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T08:54:31.915 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-13T08:54:31.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-13T08:54:58.390 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T08:54:58.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T08:54:58.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:55:01.964 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T08:55:01.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T08:55:01.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:55:16.797 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-09-13T08:55:20.381 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-13T08:55:25.804 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T08:55:25.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:55:29.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T08:55:29.385 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:55:47.778 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-13T08:55:59.789 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T08:55:59.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:56:02.845 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-13T08:56:17.857 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T08:56:17.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:56:18.056 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-13T08:56:18.660 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T08:56:18.660 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-13T08:56:18.661 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-13T08:56:18.709 INFO:teuthology.orchestra.run.smithi031.stdout:5.15.0-121-generic 2024-09-13T08:56:18.709 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T08:56:18.709 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T08:56:18.710 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T08:56:19.710 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T08:56:19.711 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T08:56:19.783 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-09-13T08:56:19.823 DEBUG:teuthology.parallel:result is None 2024-09-13T08:56:25.808 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T08:56:37.809 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T08:56:37.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T08:56:38.037 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-13T08:56:38.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T08:56:38.622 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-13T08:56:38.623 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-13T08:56:38.669 INFO:teuthology.orchestra.run.smithi060.stdout:5.15.0-121-generic 2024-09-13T08:56:38.669 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T08:56:38.669 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T08:56:38.669 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T08:56:39.670 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T08:56:39.670 DEBUG:teuthology.orchestra.run.smithi060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T08:56:39.766 INFO:teuthology.orchestra.run.smithi060.stdout:ttyS1 2024-09-13T08:56:39.798 DEBUG:teuthology.parallel:result is None 2024-09-13T08:56:39.799 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T08:56:39.807 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T08:56:39.807 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T08:56:39.810 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T08:56:39.815 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T08:56:39.852 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T08:56:39.858 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T08:56:39.858 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T08:56:39.861 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T08:56:39.879 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T08:56:39.885 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T08:56:39.886 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T08:56:39.918 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T08:56:39.933 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T08:56:39.942 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T08:56:39.949 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T08:56:39.954 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T08:56:39.956 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T08:56:39.963 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T08:56:39.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T08:56:39.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T08:56:40.014 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T08:56:40.024 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T08:56:40.024 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T08:56:40.046 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T08:56:40.060 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T08:56:40.099 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T08:56:40.146 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T08:56:40.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T08:56:40.206 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T08:56:40.212 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T08:56:40.261 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T08:56:40.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T08:56:40.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-13T08:56:40.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-09-13T08:56:40.403 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T08:56:40.411 INFO:teuthology.task.internal:Starting timer... 2024-09-13T08:56:40.411 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T08:56:40.426 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T08:56:40.435 DEBUG:teuthology.task.selinux:Excluding smithi031: OS 'ubuntu' does not support SELinux 2024-09-13T08:56:40.435 DEBUG:teuthology.task.selinux:Excluding smithi060: OS 'ubuntu' does not support SELinux 2024-09-13T08:56:40.435 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T08:56:40.435 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T08:56:40.435 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T08:56:40.435 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T08:56:40.444 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T08:56:40.456 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T08:56:40.458 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi060.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T09:04:20.213 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi060.front.sepia.ceph.com')] 2024-09-13T09:04:20.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T09:04:20.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:20.303 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-13T09:04:20.381 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-13T09:04:20.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T09:04:20.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:04:20.468 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-13T09:04:20.545 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-13T09:04:20.545 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T09:04:20.556 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T09:04:20.556 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:04:20.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:04:20.559 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T09:04:20.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:04:20.586 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T09:04:20.586 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Command line: ntpd -gq 2024-09-13T09:04:20.586 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: ---------------------------------------------------- 2024-09-13T09:04:20.586 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T09:04:20.586 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T09:04:20.586 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: corporation. Support and training for ntp-4 are 2024-09-13T09:04:20.586 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: available at https://www.nwtime.org/support 2024-09-13T09:04:20.587 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: ---------------------------------------------------- 2024-09-13T09:04:20.587 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: proto: precision = 0.067 usec (-24) 2024-09-13T09:04:20.587 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: basedate set to 2022-02-04 2024-09-13T09:04:20.587 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: gps base set to 2022-02-06 (week 2196) 2024-09-13T09:04:20.588 INFO:teuthology.orchestra.run.smithi031.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:04:20.588 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:04:20.588 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:04:20.588 INFO:teuthology.orchestra.run.smithi031.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:04:20.589 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T09:04:20.589 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T09:04:20.589 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T09:04:20.589 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Listen normally on 3 ens1f0 172.21.15.31:123 2024-09-13T09:04:20.590 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Listen normally on 4 lo [::1]:123 2024-09-13T09:04:20.590 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:caba%4]:123 2024-09-13T09:04:20.590 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:20 ntpd[18394]: Listening on routing socket on fd #22 for interface updates 2024-09-13T09:04:20.619 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T09:04:20.619 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Command line: ntpd -gq 2024-09-13T09:04:20.619 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: ---------------------------------------------------- 2024-09-13T09:04:20.619 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T09:04:20.619 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T09:04:20.619 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: corporation. Support and training for ntp-4 are 2024-09-13T09:04:20.619 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: available at https://www.nwtime.org/support 2024-09-13T09:04:20.619 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: ---------------------------------------------------- 2024-09-13T09:04:20.620 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: proto: precision = 0.051 usec (-24) 2024-09-13T09:04:20.620 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: basedate set to 2022-02-04 2024-09-13T09:04:20.620 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: gps base set to 2022-02-06 (week 2196) 2024-09-13T09:04:20.620 INFO:teuthology.orchestra.run.smithi060.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:04:20.621 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T09:04:20.621 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:04:20.621 INFO:teuthology.orchestra.run.smithi060.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T09:04:20.622 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T09:04:20.622 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T09:04:20.622 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T09:04:20.622 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Listen normally on 3 ens1f0 172.21.15.60:123 2024-09-13T09:04:20.622 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Listen normally on 4 lo [::1]:123 2024-09-13T09:04:20.623 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1560%4]:123 2024-09-13T09:04:20.623 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:20 ntpd[18424]: Listening on routing socket on fd #22 for interface updates 2024-09-13T09:04:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:21 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:21 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:21 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:21 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:21 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:21.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:21 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:21 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:21 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:21.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:21 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:21.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:21 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:21.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:21 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:21.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:21 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:21.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:21 ntpd[18424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:21.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:21 ntpd[18424]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:22 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:22 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:22 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:22 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:22 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:22 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:22.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:22 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:22.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:22 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:23.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:23.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:23.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:23.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:23 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:23.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:23.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:23 ntpd[18424]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:24.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:24 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:24.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:24 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:24.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:24 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:24.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:24 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:24.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:24 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:24.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:24 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:24.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:24 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:24.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:24 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:24.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:24 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:24.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:24 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:24.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:24 ntpd[18424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:24.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:24 ntpd[18424]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:25.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:25 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:25.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:25.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:25 ntpd[18424]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:26.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:26.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:26 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:26.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:26.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:26 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:26.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:26 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:26.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:26 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:26.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:26 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:26.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:26 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:26.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:26 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:26.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:26 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:26.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:26 ntpd[18424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:26.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:26 ntpd[18424]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:27.587 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:27.588 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:27.589 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:13 Sep 09:04:27 ntpd[18394]: ntpd: time slew -0.000918 s 2024-09-13T09:04:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:ntpd: time slew -0.000918s 2024-09-13T09:04:27.589 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T09:04:27.590 INFO:teuthology.orchestra.run.smithi031.stderr:13 Sep 09:04:27 ntpd[18394]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T09:04:27.619 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T09:04:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T09:04:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T09:04:27.620 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T09:04:27.620 INFO:teuthology.orchestra.run.smithi060.stdout:13 Sep 09:04:27 ntpd[18424]: ntpd: time slew -0.000163 s 2024-09-13T09:04:27.621 INFO:teuthology.orchestra.run.smithi060.stdout:ntpd: time slew -0.000163s 2024-09-13T09:04:27.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T09:04:27.621 INFO:teuthology.orchestra.run.smithi060.stderr:13 Sep 09:04:27 ntpd[18424]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T09:04:27.641 INFO:teuthology.orchestra.run.smithi031.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:04:27.641 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================== 2024-09-13T09:04:27.641 INFO:teuthology.orchestra.run.smithi031.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:04:27.641 INFO:teuthology.orchestra.run.smithi031.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:04:27.641 INFO:teuthology.orchestra.run.smithi031.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:04:27.641 INFO:teuthology.orchestra.run.smithi031.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:04:27.675 INFO:teuthology.orchestra.run.smithi060.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:04:27.675 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================== 2024-09-13T09:04:27.675 INFO:teuthology.orchestra.run.smithi060.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:04:27.675 INFO:teuthology.orchestra.run.smithi060.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:04:27.675 INFO:teuthology.orchestra.run.smithi060.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:04:27.675 INFO:teuthology.orchestra.run.smithi060.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:04:27.675 INFO:teuthology.run_tasks:Running task install... 2024-09-13T09:04:27.687 DEBUG:teuthology.task.install:project ceph 2024-09-13T09:04:27.687 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}} 2024-09-13T09:04:27.687 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'} 2024-09-13T09:04:27.687 INFO:teuthology.task.install:Using flavor: default 2024-09-13T09:04:27.693 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T09:04:27.693 INFO:teuthology.task.install:extra packages: [] 2024-09-13T09:04:27.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-key list | grep Ceph 2024-09-13T09:04:27.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-key list | grep Ceph 2024-09-13T09:04:27.757 INFO:teuthology.orchestra.run.smithi031.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T09:04:27.773 INFO:teuthology.orchestra.run.smithi060.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T09:04:27.785 INFO:teuthology.orchestra.run.smithi031.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T09:04:27.785 INFO:teuthology.orchestra.run.smithi031.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T09:04:27.786 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T09:04:27.787 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:04:27.801 INFO:teuthology.orchestra.run.smithi060.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T09:04:27.801 INFO:teuthology.orchestra.run.smithi060.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T09:04:27.801 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T09:04:27.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:04:27.936 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default/ 2024-09-13T09:04:27.936 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:04:27.942 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default/ 2024-09-13T09:04:27.942 INFO:teuthology.task.install.deb:Package version is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:04:28.047 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:04:28.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T09:04:28.053 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:04:28.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T09:04:28.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-get update 2024-09-13T09:04:28.068 DEBUG:teuthology.orchestra.run.smithi060:> sudo apt-get update 2024-09-13T09:04:28.234 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:04:28.290 INFO:teuthology.orchestra.run.smithi031.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T09:04:28.298 INFO:teuthology.orchestra.run.smithi060.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T09:04:28.311 INFO:teuthology.orchestra.run.smithi031.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T09:04:28.320 INFO:teuthology.orchestra.run.smithi060.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T09:04:28.332 INFO:teuthology.orchestra.run.smithi031.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T09:04:28.343 INFO:teuthology.orchestra.run.smithi060.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T09:04:28.354 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T09:04:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:04:28.372 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:04:28.373 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:04:28.373 INFO:teuthology.orchestra.run.smithi060.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-09-13T09:04:28.381 INFO:teuthology.orchestra.run.smithi031.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-09-13T09:04:28.403 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T09:04:28.437 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:04:28.444 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:04:28.519 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:04:28.525 INFO:teuthology.orchestra.run.smithi060.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:04:28.672 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 28.8 kB in 0s (68.7 kB/s) 2024-09-13T09:04:28.756 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 28.8 kB in 0s (57.8 kB/s) 2024-09-13T09:04:29.708 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-13T09:04:29.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1jammy cephadm=18.2.4-820-g48a7de7e-1jammy ceph-mds=18.2.4-820-g48a7de7e-1jammy ceph-mgr=18.2.4-820-g48a7de7e-1jammy ceph-common=18.2.4-820-g48a7de7e-1jammy ceph-fuse=18.2.4-820-g48a7de7e-1jammy ceph-test=18.2.4-820-g48a7de7e-1jammy ceph-volume=18.2.4-820-g48a7de7e-1jammy radosgw=18.2.4-820-g48a7de7e-1jammy python3-rados=18.2.4-820-g48a7de7e-1jammy python3-rgw=18.2.4-820-g48a7de7e-1jammy python3-cephfs=18.2.4-820-g48a7de7e-1jammy python3-rbd=18.2.4-820-g48a7de7e-1jammy libcephfs2=18.2.4-820-g48a7de7e-1jammy libcephfs-dev=18.2.4-820-g48a7de7e-1jammy librados2=18.2.4-820-g48a7de7e-1jammy librbd1=18.2.4-820-g48a7de7e-1jammy rbd-fuse=18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:04:29.791 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-13T09:04:29.796 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-09-13T09:04:29.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-820-g48a7de7e-1jammy cephadm=18.2.4-820-g48a7de7e-1jammy ceph-mds=18.2.4-820-g48a7de7e-1jammy ceph-mgr=18.2.4-820-g48a7de7e-1jammy ceph-common=18.2.4-820-g48a7de7e-1jammy ceph-fuse=18.2.4-820-g48a7de7e-1jammy ceph-test=18.2.4-820-g48a7de7e-1jammy ceph-volume=18.2.4-820-g48a7de7e-1jammy radosgw=18.2.4-820-g48a7de7e-1jammy python3-rados=18.2.4-820-g48a7de7e-1jammy python3-rgw=18.2.4-820-g48a7de7e-1jammy python3-cephfs=18.2.4-820-g48a7de7e-1jammy python3-rbd=18.2.4-820-g48a7de7e-1jammy libcephfs2=18.2.4-820-g48a7de7e-1jammy libcephfs-dev=18.2.4-820-g48a7de7e-1jammy librados2=18.2.4-820-g48a7de7e-1jammy librbd1=18.2.4-820-g48a7de7e-1jammy rbd-fuse=18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:04:29.877 INFO:teuthology.orchestra.run.smithi060.stdout:Reading package lists... 2024-09-13T09:04:30.001 INFO:teuthology.orchestra.run.smithi031.stdout:Building dependency tree... 2024-09-13T09:04:30.002 INFO:teuthology.orchestra.run.smithi031.stdout:Reading state information... 2024-09-13T09:04:30.096 INFO:teuthology.orchestra.run.smithi060.stdout:Building dependency tree... 2024-09-13T09:04:30.097 INFO:teuthology.orchestra.run.smithi060.stdout:Reading state information... 2024-09-13T09:04:30.196 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:04:30.196 INFO:teuthology.orchestra.run.smithi031.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:04:30.196 INFO:teuthology.orchestra.run.smithi031.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:04:30.196 INFO:teuthology.orchestra.run.smithi031.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:04:30.196 INFO:teuthology.orchestra.run.smithi031.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:04:30.197 INFO:teuthology.orchestra.run.smithi031.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:04:30.197 INFO:teuthology.orchestra.run.smithi031.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:04:30.198 INFO:teuthology.orchestra.run.smithi031.stdout:The following additional packages will be installed: 2024-09-13T09:04:30.198 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T09:04:30.198 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T09:04:30.198 INFO:teuthology.orchestra.run.smithi031.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T09:04:30.198 INFO:teuthology.orchestra.run.smithi031.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T09:04:30.199 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout:Suggested packages: 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T09:04:30.200 INFO:teuthology.orchestra.run.smithi031.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T09:04:30.201 INFO:teuthology.orchestra.run.smithi031.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T09:04:30.201 INFO:teuthology.orchestra.run.smithi031.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T09:04:30.201 INFO:teuthology.orchestra.run.smithi031.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T09:04:30.201 INFO:teuthology.orchestra.run.smithi031.stdout: python3-watchdog 2024-09-13T09:04:30.201 INFO:teuthology.orchestra.run.smithi031.stdout:Recommended packages: 2024-09-13T09:04:30.201 INFO:teuthology.orchestra.run.smithi031.stdout: btrfs-tools 2024-09-13T09:04:30.305 INFO:teuthology.orchestra.run.smithi031.stdout:The following NEW packages will be installed: 2024-09-13T09:04:30.305 INFO:teuthology.orchestra.run.smithi031.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T09:04:30.305 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T09:04:30.305 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T09:04:30.305 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-13T09:04:30.305 INFO:teuthology.orchestra.run.smithi031.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-13T09:04:30.305 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T09:04:30.306 INFO:teuthology.orchestra.run.smithi031.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T09:04:30.307 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T09:04:30.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T09:04:30.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T09:04:30.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T09:04:30.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T09:04:30.308 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T09:04:30.308 INFO:teuthology.orchestra.run.smithi031.stdout: socat xmlstarlet 2024-09-13T09:04:30.316 INFO:teuthology.orchestra.run.smithi060.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:04:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:04:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:04:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:04:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:04:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:04:30.317 INFO:teuthology.orchestra.run.smithi060.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:04:30.318 INFO:teuthology.orchestra.run.smithi060.stdout:The following additional packages will be installed: 2024-09-13T09:04:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T09:04:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T09:04:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T09:04:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T09:04:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T09:04:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T09:04:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T09:04:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T09:04:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T09:04:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T09:04:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T09:04:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T09:04:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T09:04:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T09:04:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T09:04:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T09:04:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T09:04:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T09:04:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout:Suggested packages: 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T09:04:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T09:04:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: python3-watchdog 2024-09-13T09:04:30.323 INFO:teuthology.orchestra.run.smithi060.stdout:Recommended packages: 2024-09-13T09:04:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: btrfs-tools 2024-09-13T09:04:30.366 INFO:teuthology.orchestra.run.smithi031.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T09:04:30.366 INFO:teuthology.orchestra.run.smithi031.stdout:Need to get 163 MB of archives. 2024-09-13T09:04:30.366 INFO:teuthology.orchestra.run.smithi031.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-13T09:04:30.366 INFO:teuthology.orchestra.run.smithi031.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T09:04:30.424 INFO:teuthology.orchestra.run.smithi031.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T09:04:30.432 INFO:teuthology.orchestra.run.smithi060.stdout:The following NEW packages will be installed: 2024-09-13T09:04:30.433 INFO:teuthology.orchestra.run.smithi060.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T09:04:30.433 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T09:04:30.433 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T09:04:30.433 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2024-09-13T09:04:30.433 INFO:teuthology.orchestra.run.smithi060.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-13T09:04:30.434 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T09:04:30.434 INFO:teuthology.orchestra.run.smithi060.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T09:04:30.434 INFO:teuthology.orchestra.run.smithi060.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T09:04:30.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T09:04:30.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2024-09-13T09:04:30.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T09:04:30.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T09:04:30.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T09:04:30.435 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T09:04:30.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T09:04:30.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T09:04:30.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T09:04:30.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T09:04:30.436 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T09:04:30.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T09:04:30.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T09:04:30.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T09:04:30.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T09:04:30.437 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T09:04:30.437 INFO:teuthology.orchestra.run.smithi060.stdout: socat xmlstarlet 2024-09-13T09:04:30.449 INFO:teuthology.orchestra.run.smithi031.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1jammy [3,451 kB] 2024-09-13T09:04:30.459 INFO:teuthology.orchestra.run.smithi031.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T09:04:30.464 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T09:04:30.477 INFO:teuthology.orchestra.run.smithi031.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T09:04:30.529 INFO:teuthology.orchestra.run.smithi031.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T09:04:30.533 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T09:04:30.545 INFO:teuthology.orchestra.run.smithi031.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T09:04:30.549 INFO:teuthology.orchestra.run.smithi031.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T09:04:30.549 INFO:teuthology.orchestra.run.smithi031.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T09:04:30.550 INFO:teuthology.orchestra.run.smithi031.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T09:04:30.559 INFO:teuthology.orchestra.run.smithi031.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T09:04:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T09:04:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T09:04:30.561 INFO:teuthology.orchestra.run.smithi031.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T09:04:30.567 INFO:teuthology.orchestra.run.smithi031.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T09:04:30.567 INFO:teuthology.orchestra.run.smithi031.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T09:04:30.584 INFO:teuthology.orchestra.run.smithi031.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T09:04:30.584 INFO:teuthology.orchestra.run.smithi031.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T09:04:30.585 INFO:teuthology.orchestra.run.smithi031.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T09:04:30.585 INFO:teuthology.orchestra.run.smithi031.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T09:04:30.588 INFO:teuthology.orchestra.run.smithi031.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T09:04:30.589 INFO:teuthology.orchestra.run.smithi031.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T09:04:30.589 INFO:teuthology.orchestra.run.smithi060.stdout:0 upgraded, 109 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T09:04:30.590 INFO:teuthology.orchestra.run.smithi060.stdout:Need to get 163 MB of archives. 2024-09-13T09:04:30.590 INFO:teuthology.orchestra.run.smithi060.stdout:After this operation, 746 MB of additional disk space will be used. 2024-09-13T09:04:30.590 INFO:teuthology.orchestra.run.smithi060.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-820-g48a7de7e-1jammy [3,451 kB] 2024-09-13T09:04:30.590 INFO:teuthology.orchestra.run.smithi031.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T09:04:30.590 INFO:teuthology.orchestra.run.smithi031.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T09:04:30.601 INFO:teuthology.orchestra.run.smithi031.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T09:04:30.601 INFO:teuthology.orchestra.run.smithi031.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T09:04:30.618 INFO:teuthology.orchestra.run.smithi031.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T09:04:30.619 INFO:teuthology.orchestra.run.smithi031.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T09:04:30.620 INFO:teuthology.orchestra.run.smithi031.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T09:04:30.620 INFO:teuthology.orchestra.run.smithi031.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T09:04:30.623 INFO:teuthology.orchestra.run.smithi060.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T09:04:30.626 INFO:teuthology.orchestra.run.smithi031.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T09:04:30.626 INFO:teuthology.orchestra.run.smithi031.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T09:04:30.627 INFO:teuthology.orchestra.run.smithi031.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T09:04:30.627 INFO:teuthology.orchestra.run.smithi031.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T09:04:30.628 INFO:teuthology.orchestra.run.smithi031.stdout:Get:37 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1jammy [3,108 kB] 2024-09-13T09:04:30.635 INFO:teuthology.orchestra.run.smithi031.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T09:04:30.637 INFO:teuthology.orchestra.run.smithi031.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T09:04:30.646 INFO:teuthology.orchestra.run.smithi031.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T09:04:30.648 INFO:teuthology.orchestra.run.smithi031.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T09:04:30.648 INFO:teuthology.orchestra.run.smithi031.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T09:04:30.651 INFO:teuthology.orchestra.run.smithi031.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T09:04:30.652 INFO:teuthology.orchestra.run.smithi031.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T09:04:30.652 INFO:teuthology.orchestra.run.smithi031.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T09:04:30.655 INFO:teuthology.orchestra.run.smithi031.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T09:04:30.675 INFO:teuthology.orchestra.run.smithi031.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1jammy [683 kB] 2024-09-13T09:04:30.687 INFO:teuthology.orchestra.run.smithi031.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1jammy [357 kB] 2024-09-13T09:04:30.692 INFO:teuthology.orchestra.run.smithi031.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1jammy [32.8 kB] 2024-09-13T09:04:30.693 INFO:teuthology.orchestra.run.smithi031.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1jammy [180 kB] 2024-09-13T09:04:30.696 INFO:teuthology.orchestra.run.smithi031.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1jammy [63.8 kB] 2024-09-13T09:04:30.697 INFO:teuthology.orchestra.run.smithi031.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1jammy [334 kB] 2024-09-13T09:04:30.702 INFO:teuthology.orchestra.run.smithi031.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1jammy [5,934 kB] 2024-09-13T09:04:30.709 INFO:teuthology.orchestra.run.smithi031.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T09:04:30.710 INFO:teuthology.orchestra.run.smithi031.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T09:04:30.711 INFO:teuthology.orchestra.run.smithi031.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T09:04:30.711 INFO:teuthology.orchestra.run.smithi031.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T09:04:30.712 INFO:teuthology.orchestra.run.smithi031.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T09:04:30.713 INFO:teuthology.orchestra.run.smithi031.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T09:04:30.713 INFO:teuthology.orchestra.run.smithi031.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T09:04:30.716 INFO:teuthology.orchestra.run.smithi031.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T09:04:30.716 INFO:teuthology.orchestra.run.smithi031.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T09:04:30.728 INFO:teuthology.orchestra.run.smithi031.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T09:04:30.744 INFO:teuthology.orchestra.run.smithi031.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T09:04:30.746 INFO:teuthology.orchestra.run.smithi031.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T09:04:30.747 INFO:teuthology.orchestra.run.smithi031.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T09:04:30.762 INFO:teuthology.orchestra.run.smithi031.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T09:04:30.763 INFO:teuthology.orchestra.run.smithi031.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T09:04:30.763 INFO:teuthology.orchestra.run.smithi031.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T09:04:30.764 INFO:teuthology.orchestra.run.smithi031.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T09:04:30.764 INFO:teuthology.orchestra.run.smithi031.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T09:04:30.765 INFO:teuthology.orchestra.run.smithi031.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T09:04:30.765 INFO:teuthology.orchestra.run.smithi031.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T09:04:30.780 INFO:teuthology.orchestra.run.smithi031.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T09:04:30.783 INFO:teuthology.orchestra.run.smithi031.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T09:04:30.785 INFO:teuthology.orchestra.run.smithi031.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T09:04:30.797 INFO:teuthology.orchestra.run.smithi031.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T09:04:30.798 INFO:teuthology.orchestra.run.smithi031.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T09:04:30.801 INFO:teuthology.orchestra.run.smithi031.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T09:04:30.803 INFO:teuthology.orchestra.run.smithi031.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T09:04:30.807 INFO:teuthology.orchestra.run.smithi031.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T09:04:30.810 INFO:teuthology.orchestra.run.smithi031.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T09:04:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T09:04:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1jammy [112 kB] 2024-09-13T09:04:30.813 INFO:teuthology.orchestra.run.smithi031.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1jammy [445 kB] 2024-09-13T09:04:30.814 INFO:teuthology.orchestra.run.smithi031.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T09:04:30.820 INFO:teuthology.orchestra.run.smithi031.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T09:04:30.821 INFO:teuthology.orchestra.run.smithi031.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T09:04:30.821 INFO:teuthology.orchestra.run.smithi031.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1jammy [22.0 MB] 2024-09-13T09:04:30.831 INFO:teuthology.orchestra.run.smithi031.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T09:04:30.905 INFO:teuthology.orchestra.run.smithi060.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T09:04:31.077 INFO:teuthology.orchestra.run.smithi060.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T09:04:31.091 INFO:teuthology.orchestra.run.smithi060.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T09:04:31.164 INFO:teuthology.orchestra.run.smithi060.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T09:04:31.205 INFO:teuthology.orchestra.run.smithi060.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-820-g48a7de7e-1jammy [3,108 kB] 2024-09-13T09:04:31.298 INFO:teuthology.orchestra.run.smithi031.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1jammy [5,005 kB] 2024-09-13T09:04:31.358 INFO:teuthology.orchestra.run.smithi060.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T09:04:31.358 INFO:teuthology.orchestra.run.smithi031.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1jammy [241 kB] 2024-09-13T09:04:31.361 INFO:teuthology.orchestra.run.smithi031.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1jammy [130 kB] 2024-09-13T09:04:31.362 INFO:teuthology.orchestra.run.smithi031.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1jammy [1,494 kB] 2024-09-13T09:04:31.368 INFO:teuthology.orchestra.run.smithi060.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T09:04:31.379 INFO:teuthology.orchestra.run.smithi031.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1jammy [6,085 kB] 2024-09-13T09:04:31.396 INFO:teuthology.orchestra.run.smithi060.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T09:04:31.404 INFO:teuthology.orchestra.run.smithi060.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T09:04:31.405 INFO:teuthology.orchestra.run.smithi060.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T09:04:31.406 INFO:teuthology.orchestra.run.smithi060.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T09:04:31.417 INFO:teuthology.orchestra.run.smithi060.stdout:Get:14 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-820-g48a7de7e-1jammy [683 kB] 2024-09-13T09:04:31.425 INFO:teuthology.orchestra.run.smithi060.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T09:04:31.426 INFO:teuthology.orchestra.run.smithi060.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T09:04:31.427 INFO:teuthology.orchestra.run.smithi060.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T09:04:31.440 INFO:teuthology.orchestra.run.smithi060.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T09:04:31.479 INFO:teuthology.orchestra.run.smithi060.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-820-g48a7de7e-1jammy [357 kB] 2024-09-13T09:04:31.500 INFO:teuthology.orchestra.run.smithi031.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1jammy [22.4 MB] 2024-09-13T09:04:31.510 INFO:teuthology.orchestra.run.smithi060.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-820-g48a7de7e-1jammy [32.8 kB] 2024-09-13T09:04:31.511 INFO:teuthology.orchestra.run.smithi060.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T09:04:31.512 INFO:teuthology.orchestra.run.smithi060.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-820-g48a7de7e-1jammy [180 kB] 2024-09-13T09:04:31.512 INFO:teuthology.orchestra.run.smithi060.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T09:04:31.513 INFO:teuthology.orchestra.run.smithi060.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T09:04:31.514 INFO:teuthology.orchestra.run.smithi060.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T09:04:31.514 INFO:teuthology.orchestra.run.smithi060.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T09:04:31.515 INFO:teuthology.orchestra.run.smithi060.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T09:04:31.520 INFO:teuthology.orchestra.run.smithi060.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T09:04:31.523 INFO:teuthology.orchestra.run.smithi060.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T09:04:31.523 INFO:teuthology.orchestra.run.smithi060.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T09:04:31.538 INFO:teuthology.orchestra.run.smithi060.stdout:Get:31 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-820-g48a7de7e-1jammy [63.8 kB] 2024-09-13T09:04:31.540 INFO:teuthology.orchestra.run.smithi060.stdout:Get:32 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-820-g48a7de7e-1jammy [334 kB] 2024-09-13T09:04:31.570 INFO:teuthology.orchestra.run.smithi060.stdout:Get:33 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-820-g48a7de7e-1jammy [5,934 kB] 2024-09-13T09:04:31.604 INFO:teuthology.orchestra.run.smithi060.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T09:04:31.606 INFO:teuthology.orchestra.run.smithi060.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T09:04:31.606 INFO:teuthology.orchestra.run.smithi060.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T09:04:31.607 INFO:teuthology.orchestra.run.smithi060.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T09:04:31.610 INFO:teuthology.orchestra.run.smithi060.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T09:04:31.611 INFO:teuthology.orchestra.run.smithi060.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T09:04:31.612 INFO:teuthology.orchestra.run.smithi060.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T09:04:31.686 INFO:teuthology.orchestra.run.smithi060.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T09:04:31.686 INFO:teuthology.orchestra.run.smithi060.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T09:04:31.687 INFO:teuthology.orchestra.run.smithi060.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T09:04:31.767 INFO:teuthology.orchestra.run.smithi060.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T09:04:31.769 INFO:teuthology.orchestra.run.smithi060.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T09:04:31.773 INFO:teuthology.orchestra.run.smithi060.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T09:04:31.775 INFO:teuthology.orchestra.run.smithi060.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T09:04:31.778 INFO:teuthology.orchestra.run.smithi060.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T09:04:31.779 INFO:teuthology.orchestra.run.smithi060.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T09:04:31.784 INFO:teuthology.orchestra.run.smithi060.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T09:04:31.848 INFO:teuthology.orchestra.run.smithi060.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T09:04:31.850 INFO:teuthology.orchestra.run.smithi060.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T09:04:31.854 INFO:teuthology.orchestra.run.smithi060.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T09:04:32.017 INFO:teuthology.orchestra.run.smithi060.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T09:04:32.019 INFO:teuthology.orchestra.run.smithi060.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T09:04:32.021 INFO:teuthology.orchestra.run.smithi060.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T09:04:32.022 INFO:teuthology.orchestra.run.smithi060.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T09:04:32.024 INFO:teuthology.orchestra.run.smithi060.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T09:04:32.025 INFO:teuthology.orchestra.run.smithi060.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T09:04:32.025 INFO:teuthology.orchestra.run.smithi060.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T09:04:32.058 INFO:teuthology.orchestra.run.smithi031.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1jammy [14.2 kB] 2024-09-13T09:04:32.058 INFO:teuthology.orchestra.run.smithi031.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1jammy [834 kB] 2024-09-13T09:04:32.062 INFO:teuthology.orchestra.run.smithi060.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T09:04:32.063 INFO:teuthology.orchestra.run.smithi060.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T09:04:32.063 INFO:teuthology.orchestra.run.smithi060.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-820-g48a7de7e-1jammy [112 kB] 2024-09-13T09:04:32.065 INFO:teuthology.orchestra.run.smithi060.stdout:Get:64 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-820-g48a7de7e-1jammy [445 kB] 2024-09-13T09:04:32.072 INFO:teuthology.orchestra.run.smithi031.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1jammy [2,159 kB] 2024-09-13T09:04:32.074 INFO:teuthology.orchestra.run.smithi060.stdout:Get:65 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-820-g48a7de7e-1jammy [22.0 MB] 2024-09-13T09:04:32.102 INFO:teuthology.orchestra.run.smithi060.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T09:04:32.107 INFO:teuthology.orchestra.run.smithi031.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1jammy [240 kB] 2024-09-13T09:04:32.111 INFO:teuthology.orchestra.run.smithi031.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1jammy [145 kB] 2024-09-13T09:04:32.113 INFO:teuthology.orchestra.run.smithi031.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1jammy [2,233 kB] 2024-09-13T09:04:32.138 INFO:teuthology.orchestra.run.smithi031.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1jammy [8,625 kB] 2024-09-13T09:04:32.147 INFO:teuthology.orchestra.run.smithi060.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T09:04:32.151 INFO:teuthology.orchestra.run.smithi060.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T09:04:32.152 INFO:teuthology.orchestra.run.smithi060.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T09:04:32.211 INFO:teuthology.orchestra.run.smithi060.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T09:04:32.211 INFO:teuthology.orchestra.run.smithi060.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T09:04:32.213 INFO:teuthology.orchestra.run.smithi060.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T09:04:32.214 INFO:teuthology.orchestra.run.smithi060.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T09:04:32.215 INFO:teuthology.orchestra.run.smithi060.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T09:04:32.216 INFO:teuthology.orchestra.run.smithi060.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T09:04:32.267 INFO:teuthology.orchestra.run.smithi060.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T09:04:32.268 INFO:teuthology.orchestra.run.smithi060.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T09:04:32.274 INFO:teuthology.orchestra.run.smithi060.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T09:04:32.336 INFO:teuthology.orchestra.run.smithi031.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1jammy [14.3 kB] 2024-09-13T09:04:32.336 INFO:teuthology.orchestra.run.smithi031.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1jammy [42.8 MB] 2024-09-13T09:04:32.348 INFO:teuthology.orchestra.run.smithi060.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T09:04:32.351 INFO:teuthology.orchestra.run.smithi060.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T09:04:32.353 INFO:teuthology.orchestra.run.smithi060.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T09:04:32.361 INFO:teuthology.orchestra.run.smithi060.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T09:04:32.368 INFO:teuthology.orchestra.run.smithi060.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T09:04:32.379 INFO:teuthology.orchestra.run.smithi060.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T09:04:32.386 INFO:teuthology.orchestra.run.smithi060.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T09:04:32.429 INFO:teuthology.orchestra.run.smithi060.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T09:04:32.433 INFO:teuthology.orchestra.run.smithi060.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T09:04:32.451 INFO:teuthology.orchestra.run.smithi060.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T09:04:32.513 INFO:teuthology.orchestra.run.smithi060.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T09:04:32.513 INFO:teuthology.orchestra.run.smithi060.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T09:04:33.155 INFO:teuthology.orchestra.run.smithi060.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-820-g48a7de7e-1jammy [5,005 kB] 2024-09-13T09:04:33.246 INFO:teuthology.orchestra.run.smithi060.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-820-g48a7de7e-1jammy [241 kB] 2024-09-13T09:04:33.251 INFO:teuthology.orchestra.run.smithi060.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-820-g48a7de7e-1jammy [130 kB] 2024-09-13T09:04:33.254 INFO:teuthology.orchestra.run.smithi060.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-820-g48a7de7e-1jammy [1,494 kB] 2024-09-13T09:04:33.325 INFO:teuthology.orchestra.run.smithi060.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-820-g48a7de7e-1jammy [6,085 kB] 2024-09-13T09:04:33.542 INFO:teuthology.orchestra.run.smithi060.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-820-g48a7de7e-1jammy [22.4 MB] 2024-09-13T09:04:33.663 INFO:teuthology.orchestra.run.smithi031.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1jammy [122 kB] 2024-09-13T09:04:33.666 INFO:teuthology.orchestra.run.smithi031.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1jammy [38.3 kB] 2024-09-13T09:04:33.667 INFO:teuthology.orchestra.run.smithi031.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1jammy [11.7 MB] 2024-09-13T09:04:33.931 INFO:teuthology.orchestra.run.smithi031.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1jammy [93.1 kB] 2024-09-13T09:04:34.150 INFO:teuthology.orchestra.run.smithi060.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-820-g48a7de7e-1jammy [14.2 kB] 2024-09-13T09:04:34.151 INFO:teuthology.orchestra.run.smithi060.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-820-g48a7de7e-1jammy [834 kB] 2024-09-13T09:04:34.162 INFO:teuthology.orchestra.run.smithi060.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-820-g48a7de7e-1jammy [2,159 kB] 2024-09-13T09:04:34.190 INFO:teuthology.orchestra.run.smithi060.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-820-g48a7de7e-1jammy [240 kB] 2024-09-13T09:04:34.193 INFO:teuthology.orchestra.run.smithi060.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-820-g48a7de7e-1jammy [145 kB] 2024-09-13T09:04:34.195 INFO:teuthology.orchestra.run.smithi060.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-820-g48a7de7e-1jammy [2,233 kB] 2024-09-13T09:04:34.219 INFO:teuthology.orchestra.run.smithi060.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-820-g48a7de7e-1jammy [8,625 kB] 2024-09-13T09:04:34.351 INFO:teuthology.orchestra.run.smithi060.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-820-g48a7de7e-1jammy [14.3 kB] 2024-09-13T09:04:34.351 INFO:teuthology.orchestra.run.smithi060.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-820-g48a7de7e-1jammy [42.8 MB] 2024-09-13T09:04:34.491 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 163 MB in 4s (45.0 MB/s) 2024-09-13T09:04:34.601 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T09:04:34.681 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T09:04:34.686 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T09:04:34.759 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T09:04:35.078 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T09:04:35.105 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T09:04:35.170 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T09:04:35.211 INFO:teuthology.orchestra.run.smithi060.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-820-g48a7de7e-1jammy [122 kB] 2024-09-13T09:04:35.279 INFO:teuthology.orchestra.run.smithi060.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-820-g48a7de7e-1jammy [38.3 kB] 2024-09-13T09:04:35.280 INFO:teuthology.orchestra.run.smithi060.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-820-g48a7de7e-1jammy [11.7 MB] 2024-09-13T09:04:35.438 INFO:teuthology.orchestra.run.smithi060.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-820-g48a7de7e-1jammy [93.1 kB] 2024-09-13T09:04:35.523 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T09:04:35.550 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T09:04:35.590 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T09:04:35.934 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T09:04:35.981 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T09:04:36.001 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T09:04:36.033 INFO:teuthology.orchestra.run.smithi060.stdout:Fetched 163 MB in 5s (32.5 MB/s) 2024-09-13T09:04:36.129 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T09:04:36.185 INFO:teuthology.orchestra.run.smithi060.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T09:04:36.190 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T09:04:36.271 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T09:04:36.362 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T09:04:36.389 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:04:36.438 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:04:36.599 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T09:04:36.626 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T09:04:36.665 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T09:04:36.841 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T09:04:36.867 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:04:36.907 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:04:36.969 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T09:04:36.997 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T09:04:37.027 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T09:04:37.243 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T09:04:37.270 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:04:37.310 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:04:37.374 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T09:04:37.401 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T09:04:37.441 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T09:04:37.654 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T09:04:37.681 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T09:04:37.712 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T09:04:37.844 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T09:04:37.871 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:04:37.919 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:04:38.049 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librados2. 2024-09-13T09:04:38.075 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:38.115 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:38.331 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T09:04:38.358 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:04:38.397 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:04:38.560 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librbd1. 2024-09-13T09:04:38.586 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../009-librbd1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:38.626 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:38.872 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T09:04:38.897 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T09:04:38.967 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:04:39.079 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T09:04:39.105 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:39.145 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:39.304 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T09:04:39.331 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T09:04:39.370 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T09:04:39.431 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rados. 2024-09-13T09:04:39.458 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../011-python3-rados_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:39.497 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:39.707 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package librados2. 2024-09-13T09:04:39.734 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../008-librados2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:39.782 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:39.783 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T09:04:39.810 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:04:39.842 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:40.077 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T09:04:40.093 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:40.136 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:40.227 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package librbd1. 2024-09-13T09:04:40.253 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../009-librbd1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:40.293 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:40.430 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T09:04:40.456 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:04:40.496 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:40.704 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T09:04:40.731 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../010-libcephfs2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:40.762 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:40.808 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T09:04:40.834 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T09:04:40.874 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T09:04:41.023 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-rados. 2024-09-13T09:04:41.050 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../011-python3-rados_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:41.099 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T09:04:41.228 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T09:04:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T09:04:41.418 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T09:04:41.445 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:04:41.485 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:41.571 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T09:04:41.598 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:41.646 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:41.720 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T09:04:41.737 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../013-python3-cephfs_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:41.779 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:42.048 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T09:04:42.049 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T09:04:42.075 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../014-python3-ceph-common_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:04:42.075 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T09:04:42.107 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:42.115 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T09:04:42.427 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T09:04:42.454 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T09:04:42.459 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librgw2. 2024-09-13T09:04:42.487 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:42.493 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T09:04:42.534 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:42.788 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T09:04:42.815 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T09:04:42.846 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T09:04:43.028 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T09:04:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:43.078 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:43.115 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T09:04:43.142 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../017-python3-rbd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:43.190 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:43.406 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T09:04:43.432 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T09:04:43.472 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T09:04:43.493 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T09:04:43.509 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T09:04:43.735 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T09:04:43.783 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T09:04:43.810 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:43.850 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:44.147 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package librgw2. 2024-09-13T09:04:44.173 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../019-librgw2_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:44.213 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:44.220 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-common. 2024-09-13T09:04:44.246 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:44.278 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:44.741 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T09:04:44.757 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:44.807 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:45.110 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T09:04:45.137 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T09:04:45.177 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T09:04:45.488 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T09:04:45.515 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:45.555 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:45.583 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-base. 2024-09-13T09:04:45.610 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:45.676 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:45.992 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-common. 2024-09-13T09:04:46.018 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../023-ceph-common_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:46.058 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:46.237 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T09:04:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T09:04:46.303 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T09:04:46.615 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T09:04:46.641 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T09:04:46.681 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T09:04:47.009 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T09:04:47.036 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T09:04:47.076 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T09:04:47.320 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T09:04:47.347 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T09:04:47.372 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-base. 2024-09-13T09:04:47.378 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T09:04:47.399 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../024-ceph-base_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:47.448 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:47.656 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T09:04:47.682 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T09:04:47.731 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T09:04:48.011 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T09:04:48.034 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T09:04:48.037 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T09:04:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T09:04:48.077 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T09:04:48.100 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T09:04:48.380 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T09:04:48.406 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T09:04:48.463 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T09:04:48.469 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-portend. 2024-09-13T09:04:48.497 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-13T09:04:48.544 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T09:04:48.783 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T09:04:48.810 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T09:04:48.849 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T09:04:49.144 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T09:04:49.171 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T09:04:49.211 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T09:04:49.247 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T09:04:49.274 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T09:04:49.322 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T09:04:49.489 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T09:04:49.516 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T09:04:49.555 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T09:04:49.666 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T09:04:49.693 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T09:04:49.775 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T09:04:49.859 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T09:04:49.885 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T09:04:49.925 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T09:04:50.136 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T09:04:50.164 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T09:04:50.203 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T09:04:50.228 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-portend. 2024-09-13T09:04:50.255 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2024-09-13T09:04:50.294 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T09:04:50.506 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T09:04:50.533 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T09:04:50.556 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T09:04:50.572 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T09:04:50.582 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T09:04:50.614 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T09:04:50.858 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T09:04:50.886 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T09:04:50.900 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T09:04:50.925 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T09:04:50.927 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T09:04:50.967 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T09:04:51.194 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-mako. 2024-09-13T09:04:51.221 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T09:04:51.261 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T09:04:51.304 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T09:04:51.320 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T09:04:51.357 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T09:04:51.564 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T09:04:51.580 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T09:04:51.613 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:04:51.632 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T09:04:51.659 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T09:04:51.699 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T09:04:51.900 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T09:04:51.927 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T09:04:51.968 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T09:04:52.002 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T09:04:52.028 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T09:04:52.068 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T09:04:52.269 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-webob. 2024-09-13T09:04:52.296 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T09:04:52.335 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T09:04:52.371 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-mako. 2024-09-13T09:04:52.398 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T09:04:52.438 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T09:04:52.688 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T09:04:52.715 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T09:04:52.757 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T09:04:52.758 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T09:04:52.773 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T09:04:52.807 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:04:53.060 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T09:04:53.085 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T09:04:53.086 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T09:04:53.101 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T09:04:53.126 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T09:04:53.144 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T09:04:53.429 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-paste. 2024-09-13T09:04:53.447 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-webob. 2024-09-13T09:04:53.456 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T09:04:53.473 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T09:04:53.496 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T09:04:53.513 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T09:04:53.849 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T09:04:53.866 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T09:04:53.876 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T09:04:53.893 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T09:04:53.915 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T09:04:53.937 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T09:04:54.377 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T09:04:54.404 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T09:04:54.436 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T09:04:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T09:04:54.463 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T09:04:54.502 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T09:04:54.722 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T09:04:54.747 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-paste. 2024-09-13T09:04:54.748 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T09:04:54.774 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T09:04:54.779 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T09:04:54.814 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T09:04:55.057 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T09:04:55.084 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T09:04:55.124 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T09:04:55.134 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T09:04:55.150 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T09:04:55.184 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T09:04:55.470 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T09:04:55.479 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T09:04:55.497 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T09:04:55.506 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T09:04:55.537 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T09:04:55.545 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T09:04:55.840 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T09:04:55.867 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T09:04:55.873 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-idna. 2024-09-13T09:04:55.900 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T09:04:55.906 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T09:04:55.940 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T09:04:56.218 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T09:04:56.243 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T09:04:56.245 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T09:04:56.270 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T09:04:56.284 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T09:04:56.309 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T09:04:56.613 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-requests. 2024-09-13T09:04:56.631 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T09:04:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T09:04:56.658 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T09:04:56.679 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T09:04:56.697 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T09:04:56.941 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T09:04:56.967 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T09:04:56.992 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-idna. 2024-09-13T09:04:57.007 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T09:04:57.019 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../049-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T09:04:57.050 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T09:04:57.302 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T09:04:57.318 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:04:57.352 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:57.378 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T09:04:57.405 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T09:04:57.445 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T09:04:57.740 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T09:04:57.756 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-requests. 2024-09-13T09:04:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:57.784 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T09:04:57.806 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:57.823 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T09:04:58.135 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T09:04:58.151 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T09:04:58.162 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T09:04:58.178 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T09:04:58.201 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T09:04:58.209 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T09:04:58.479 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T09:04:58.504 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T09:04:58.506 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T09:04:58.532 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:04:58.545 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T09:04:58.571 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:58.890 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T09:04:58.917 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:58.943 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T09:04:58.956 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:58.970 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:59.001 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:59.287 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T09:04:59.314 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../055-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T09:04:59.353 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T09:04:59.418 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T09:04:59.445 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:04:59.485 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:04:59.707 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T09:04:59.734 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../056-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T09:04:59.773 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T09:05:00.072 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T09:05:00.099 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:00.118 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T09:05:00.138 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:00.145 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../057-ceph-mgr_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:00.184 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:00.513 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T09:05:00.540 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../058-ceph-mon_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:00.571 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:01.167 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph. 2024-09-13T09:05:01.175 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T09:05:01.182 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../060-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:01.202 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../059-ceph-osd_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:01.216 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:01.241 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:01.486 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T09:05:01.513 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:01.553 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:01.931 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T09:05:01.948 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:01.989 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:02.312 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph. 2024-09-13T09:05:02.328 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../060-ceph_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:02.360 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephadm. 2024-09-13T09:05:02.362 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:02.386 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../063-cephadm_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:02.426 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:02.623 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T09:05:02.650 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../061-ceph-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:02.689 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:02.712 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T09:05:02.739 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T09:05:02.779 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T09:05:03.085 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T09:05:03.090 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T09:05:03.106 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T09:05:03.111 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../062-ceph-mds_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:03.140 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T09:05:03.151 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:04.255 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package cephadm. 2024-09-13T09:05:04.282 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../063-cephadm_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:04.371 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:04.808 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T09:05:04.834 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../064-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T09:05:04.874 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T09:05:05.145 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-babel. 2024-09-13T09:05:05.173 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T09:05:05.177 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T09:05:05.193 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../065-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T09:05:05.212 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T09:05:05.227 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T09:05:05.548 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T09:05:05.576 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T09:05:05.615 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T09:05:05.994 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T09:05:06.021 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:06.060 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:06.266 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-babel. 2024-09-13T09:05:06.293 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../066-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T09:05:06.332 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T09:05:06.422 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T09:05:06.449 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T09:05:06.488 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T09:05:06.736 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T09:05:06.763 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../067-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T09:05:06.800 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-py. 2024-09-13T09:05:06.802 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T09:05:06.827 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-13T09:05:06.866 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T09:05:07.122 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T09:05:07.149 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../068-ceph-mgr-cephadm_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:07.188 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:07.203 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T09:05:07.231 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T09:05:07.270 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T09:05:07.526 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T09:05:07.531 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-toml. 2024-09-13T09:05:07.553 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../069-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T09:05:07.558 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-13T09:05:07.597 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T09:05:07.617 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T09:05:07.900 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T09:05:07.920 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-py. 2024-09-13T09:05:07.928 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T09:05:07.948 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../070-python3-py_1.10.0-1_all.deb ... 2024-09-13T09:05:07.966 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T09:05:07.987 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T09:05:08.257 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T09:05:08.273 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../071-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T09:05:08.287 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T09:05:08.303 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T09:05:08.307 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T09:05:08.345 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T09:05:08.593 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-toml. 2024-09-13T09:05:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../072-python3-toml_0.10.2-1_all.deb ... 2024-09-13T09:05:08.648 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-routes. 2024-09-13T09:05:08.660 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T09:05:08.676 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T09:05:08.715 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T09:05:08.963 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T09:05:08.990 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../073-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T09:05:09.029 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T09:05:09.077 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T09:05:09.105 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:09.143 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:09.375 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T09:05:09.403 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../074-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T09:05:09.441 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T09:05:09.787 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-routes. 2024-09-13T09:05:09.814 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../075-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T09:05:09.853 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T09:05:10.198 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T09:05:10.225 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../076-ceph-mgr-dashboard_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:10.264 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:10.424 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T09:05:10.451 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T09:05:10.490 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T09:05:11.027 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T09:05:11.055 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T09:05:11.094 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T09:05:11.453 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T09:05:11.470 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../077-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T09:05:11.472 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T09:05:11.500 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T09:05:11.511 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T09:05:11.539 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T09:05:11.817 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T09:05:11.844 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T09:05:11.883 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T09:05:12.025 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T09:05:12.051 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../078-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T09:05:12.090 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T09:05:12.461 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T09:05:12.489 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../079-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T09:05:12.527 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T09:05:12.600 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T09:05:12.617 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:12.658 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:12.833 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T09:05:12.858 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../080-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T09:05:12.897 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T09:05:13.589 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T09:05:13.617 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../081-ceph-mgr-diskprediction-local_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:13.628 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T09:05:13.657 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T09:05:13.657 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:13.695 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T09:05:14.124 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T09:05:14.152 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T09:05:14.191 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T09:05:14.536 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T09:05:14.564 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T09:05:14.575 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T09:05:14.603 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T09:05:14.603 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../082-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T09:05:14.633 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T09:05:14.889 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T09:05:14.917 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-13T09:05:14.956 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T09:05:15.140 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T09:05:15.167 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../083-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T09:05:15.206 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T09:05:15.492 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T09:05:15.501 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T09:05:15.518 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../084-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T09:05:15.520 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T09:05:15.551 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T09:05:15.559 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T09:05:15.846 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T09:05:15.854 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T09:05:15.874 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../085-python3-rsa_4.8-1_all.deb ... 2024-09-13T09:05:15.881 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T09:05:15.904 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T09:05:15.920 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T09:05:16.225 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T09:05:16.232 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T09:05:16.252 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../086-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T09:05:16.259 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T09:05:16.291 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T09:05:16.298 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T09:05:16.618 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T09:05:16.619 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T09:05:16.646 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T09:05:16.647 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T09:05:16.685 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T09:05:16.686 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T09:05:16.956 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T09:05:16.972 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../088-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T09:05:17.006 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T09:05:17.285 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T09:05:17.312 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../089-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T09:05:17.332 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T09:05:17.349 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:17.350 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T09:05:17.390 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:17.634 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T09:05:17.651 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T09:05:17.701 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T09:05:17.965 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T09:05:17.982 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:18.023 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:18.046 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T09:05:18.074 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T09:05:18.112 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T09:05:18.318 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T09:05:18.346 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../091-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T09:05:18.376 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T09:05:18.382 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package jq. 2024-09-13T09:05:18.410 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T09:05:18.456 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T09:05:18.684 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package socat. 2024-09-13T09:05:18.687 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T09:05:18.701 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T09:05:18.715 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../092-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T09:05:18.734 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T09:05:18.754 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T09:05:19.015 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package jq. 2024-09-13T09:05:19.029 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T09:05:19.031 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../093-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T09:05:19.046 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T09:05:19.073 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T09:05:19.087 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T09:05:19.318 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package socat. 2024-09-13T09:05:19.334 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../094-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T09:05:19.376 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T09:05:19.433 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-test. 2024-09-13T09:05:19.450 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../096-ceph-test_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:19.483 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:19.663 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T09:05:19.690 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T09:05:19.729 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T09:05:20.125 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-test. 2024-09-13T09:05:20.142 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../096-ceph-test_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:20.183 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:21.463 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T09:05:21.491 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:21.529 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:21.808 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T09:05:21.824 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:21.866 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:22.105 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T09:05:22.133 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-820-g48a7de7e-1jammy_all.deb ... 2024-09-13T09:05:22.171 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:22.220 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T09:05:22.247 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T09:05:22.286 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T09:05:22.475 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T09:05:22.503 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../098-libcephfs-dev_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:22.541 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:22.649 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T09:05:22.666 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T09:05:22.707 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T09:05:22.928 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T09:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../099-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T09:05:22.994 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T09:05:23.144 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T09:05:23.173 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T09:05:23.210 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T09:05:23.342 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T09:05:23.370 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../100-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T09:05:23.408 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T09:05:23.514 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T09:05:23.543 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T09:05:23.580 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T09:05:23.870 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T09:05:23.898 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../101-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T09:05:23.917 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T09:05:23.936 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T09:05:23.945 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T09:05:23.983 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T09:05:24.282 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T09:05:24.309 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../102-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T09:05:24.348 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T09:05:24.471 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T09:05:24.499 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T09:05:24.537 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T09:05:24.676 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T09:05:24.704 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../103-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T09:05:24.742 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T09:05:24.857 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T09:05:24.885 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T09:05:24.924 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T09:05:25.209 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T09:05:25.226 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T09:05:25.267 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T09:05:25.347 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T09:05:25.375 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../104-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T09:05:25.413 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T09:05:25.725 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T09:05:25.753 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T09:05:25.791 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T09:05:25.980 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package radosgw. 2024-09-13T09:05:26.009 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../107-radosgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:26.038 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:26.060 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T09:05:26.077 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T09:05:26.118 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T09:05:26.791 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T09:05:26.808 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:26.814 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package radosgw. 2024-09-13T09:05:26.842 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../107-radosgw_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:26.849 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:26.872 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:27.163 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T09:05:27.412 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T09:05:27.521 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T09:05:27.592 INFO:teuthology.orchestra.run.smithi060.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T09:05:27.609 INFO:teuthology.orchestra.run.smithi060.stdout:Preparing to unpack .../108-rbd-fuse_18.2.4-820-g48a7de7e-1jammy_amd64.deb ... 2024-09-13T09:05:27.650 INFO:teuthology.orchestra.run.smithi060.stdout:Unpacking rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:27.769 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T09:05:27.963 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T09:05:28.098 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T09:05:28.247 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T09:05:28.373 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T09:05:28.511 INFO:teuthology.orchestra.run.smithi031.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T09:05:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:28.639 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T09:05:28.955 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T09:05:29.073 INFO:teuthology.orchestra.run.smithi031.stdout:Adding system user cephadm....done 2024-09-13T09:05:29.229 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T09:05:29.366 INFO:teuthology.orchestra.run.smithi060.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T09:05:29.442 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:29.497 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T09:05:29.749 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T09:05:29.875 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T09:05:29.975 INFO:teuthology.orchestra.run.smithi060.stdout:Adding system user cephadm....done 2024-09-13T09:05:30.134 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T09:05:30.410 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T09:05:30.458 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T09:05:30.528 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T09:05:30.829 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T09:05:30.830 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T09:05:31.158 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T09:05:31.198 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T09:05:31.410 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T09:05:31.466 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T09:05:31.713 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T09:05:31.742 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T09:05:31.956 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T09:05:32.002 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T09:05:32.066 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T09:05:32.270 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T09:05:32.418 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T09:05:32.529 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:32.786 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T09:05:32.905 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T09:05:33.006 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T09:05:33.046 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T09:05:33.257 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T09:05:33.330 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T09:05:33.383 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T09:05:33.565 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T09:05:33.751 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T09:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T09:05:33.877 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T09:05:34.092 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-ceph-argparse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:34.145 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T09:05:34.361 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T09:05:34.421 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T09:05:34.486 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T09:05:34.714 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T09:05:34.753 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T09:05:34.888 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T09:05:35.239 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T09:05:35.349 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T09:05:35.457 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T09:05:35.625 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T09:05:35.742 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T09:05:35.892 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T09:05:36.177 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T09:05:36.210 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T09:05:36.303 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T09:05:36.596 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T09:05:36.963 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T09:05:36.988 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T09:05:37.222 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T09:05:37.223 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T09:05:37.323 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T09:05:37.433 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T09:05:37.624 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T09:05:37.767 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T09:05:37.942 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T09:05:38.168 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T09:05:38.337 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T09:05:38.428 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T09:05:38.572 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T09:05:38.672 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:38.813 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T09:05:38.965 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T09:05:39.114 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T09:05:39.292 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T09:05:39.441 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T09:05:39.692 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T09:05:39.818 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:40.119 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T09:05:40.454 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T09:05:40.513 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:05:40.647 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T09:05:40.924 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T09:05:41.050 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T09:05:41.175 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T09:05:41.477 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T09:05:41.692 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:05:41.753 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T09:05:41.818 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T09:05:42.004 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:05:42.086 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T09:05:42.134 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T09:05:42.212 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T09:05:42.256 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T09:05:42.339 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T09:05:42.557 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T09:05:42.623 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T09:05:42.817 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T09:05:42.883 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T09:05:43.077 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T09:05:43.142 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:05:43.268 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T09:05:43.369 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:05:43.394 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T09:05:43.628 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T09:05:43.704 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T09:05:43.896 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T09:05:44.022 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T09:05:44.055 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T09:05:44.315 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T09:05:44.331 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T09:05:44.440 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T09:05:44.582 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T09:05:44.766 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T09:05:44.842 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T09:05:45.068 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T09:05:45.151 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T09:05:45.277 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T09:05:45.318 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T09:05:45.554 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T09:05:45.680 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T09:05:45.680 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T09:05:45.955 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T09:05:46.081 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T09:05:46.214 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T09:05:46.499 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T09:05:46.524 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T09:05:46.766 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T09:05:46.775 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T09:05:47.017 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:05:47.152 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T09:05:47.249 INFO:teuthology.orchestra.run.smithi060.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T09:05:47.481 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T09:05:47.503 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T09:05:47.629 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T09:05:47.787 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T09:05:48.006 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T09:05:48.063 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T09:05:48.172 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T09:05:48.274 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T09:05:48.448 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T09:05:48.508 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T09:05:48.634 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T09:05:48.733 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T09:05:48.960 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T09:05:48.984 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T09:05:49.286 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T09:05:49.352 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T09:05:49.571 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T09:05:49.653 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T09:05:49.697 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T09:05:49.965 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T09:05:49.988 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T09:05:50.115 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:50.241 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:50.241 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T09:05:50.366 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:50.492 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:50.500 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T09:05:50.618 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:50.830 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T09:05:50.868 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T09:05:51.098 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:51.153 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T09:05:51.249 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:51.375 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:51.514 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T09:05:51.640 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up librados2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:51.749 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up librgw2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:51.803 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:05:51.804 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:05:51.850 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libsqlite3-mod-ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:51.960 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libcephfs2 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:52.086 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libradosstriper1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:52.293 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:52.297 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T09:05:52.419 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:52.553 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T09:05:52.574 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up librbd1 (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:52.725 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-modules-core (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:52.851 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:53.071 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:05:53.071 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T09:05:53.553 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up libcephfs-dev (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:53.679 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-rados (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:53.780 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T09:05:53.791 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:53.917 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:54.043 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:54.169 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:54.303 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:54.830 INFO:teuthology.orchestra.run.smithi031.stdout:Adding group ceph....done 2024-09-13T09:05:55.059 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-rbd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:55.185 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up rbd-fuse (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:55.215 INFO:teuthology.orchestra.run.smithi031.stdout:Adding system user ceph....done 2024-09-13T09:05:55.311 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-rgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:55.416 INFO:teuthology.orchestra.run.smithi031.stdout:Setting system user ceph properties....done 2024-09-13T09:05:55.434 INFO:teuthology.orchestra.run.smithi031.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T09:05:55.437 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up python3-cephfs (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:55.572 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-common (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:55.582 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T09:05:55.929 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T09:05:56.082 INFO:teuthology.orchestra.run.smithi060.stdout:Adding group ceph....done 2024-09-13T09:05:56.387 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:56.434 INFO:teuthology.orchestra.run.smithi060.stdout:Adding system user ceph....done 2024-09-13T09:05:56.513 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:56.602 INFO:teuthology.orchestra.run.smithi060.stdout:Setting system user ceph properties....done 2024-09-13T09:05:56.620 INFO:teuthology.orchestra.run.smithi060.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T09:05:56.763 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T09:05:56.953 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:05:56.953 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:05:57.081 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T09:05:57.407 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:57.558 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-test (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:57.659 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up radosgw (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:57.766 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T09:05:58.088 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:05:58.088 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T09:05:58.225 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:58.471 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:05:58.471 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:05:58.527 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-base (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:58.875 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T09:05:58.927 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:59.162 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:05:59.162 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:05:59.371 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mds (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:59.607 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:05:59.607 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T09:05:59.628 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:05:59.918 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:05:59.918 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:06:00.098 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:00.316 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:06:00.316 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T09:06:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:00.531 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:00.681 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:00.766 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-osd (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:00.900 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:06:00.900 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:06:01.064 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:06:01.065 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T09:06:01.349 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:01.500 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:01.559 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-k8sevents (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:01.626 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:01.685 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:01.777 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:01.836 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mon (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:02.054 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:06:02.055 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T09:06:02.131 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T09:06:02.513 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-cephadm (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:02.672 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:02.798 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-mgr-dashboard (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:02.950 INFO:teuthology.orchestra.run.smithi060.stdout:Setting up ceph-volume (18.2.4-820-g48a7de7e-1jammy) ... 2024-09-13T09:06:03.248 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T09:06:03.311 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T09:06:03.758 INFO:teuthology.orchestra.run.smithi031.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T09:06:03.765 DEBUG:teuthology.parallel:result is None 2024-09-13T09:06:04.222 INFO:teuthology.orchestra.run.smithi060.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T09:06:04.690 INFO:teuthology.orchestra.run.smithi060.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T09:06:04.695 DEBUG:teuthology.parallel:result is None 2024-09-13T09:06:04.696 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:06:04.841 DEBUG:teuthology.orchestra.run.smithi031:> dpkg-query -W -f '${Version}' ceph 2024-09-13T09:06:04.861 INFO:teuthology.orchestra.run.smithi031.stdout:18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:06:04.862 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:06:04.862 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1jammy is installed. 2024-09-13T09:06:04.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T09:06:05.003 DEBUG:teuthology.orchestra.run.smithi060:> dpkg-query -W -f '${Version}' ceph 2024-09-13T09:06:05.022 INFO:teuthology.orchestra.run.smithi060.stdout:18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:06:05.023 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820-g48a7de7e-1jammy 2024-09-13T09:06:05.023 INFO:teuthology.task.install:The correct ceph version 18.2.4-820-g48a7de7e-1jammy is installed. 2024-09-13T09:06:05.025 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T09:06:05.025 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:05.025 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:06:05.040 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:05.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T09:06:05.077 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T09:06:05.077 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:05.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:06:05.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:06:05.152 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:05.152 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T09:06:05.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T09:06:05.221 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T09:06:05.222 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:05.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:06:05.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:06:05.291 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:05.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T09:06:05.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T09:06:05.360 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T09:06:05.360 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:05.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:06:05.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:06:05.427 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:05.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T09:06:05.441 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T09:06:05.496 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T09:06:05.576 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T09:06:05.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-09-13T09:06:05.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /var/log/ceph 2024-09-13T09:06:05.592 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T09:06:05.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:06:05.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T09:06:05.649 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T09:06:05.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:06:05.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T09:06:05.714 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T09:06:05.714 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T09:06:05.714 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7901010', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 248, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'sleep_before_teardown': 0, 'subset': '116/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN69Fh17vuaP8Jp0Ac+y2fLkte/crXDtelNrBYCTB0GJddGdwfnpR8+P0RNLu56vIql0leV37IuCRYMZCOomtCY=', 'smithi060.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLvUXqVvFJmICKsPIqTyb3eS2g2iw1eavX67ABEGJpBOc85TRch+6fj5gQAV9489B8H96uTo8+SOys1rn0yKFBc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T09:06:05.715 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:06:05.745 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T09:06:05.753 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:06:05.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T09:06:05.809 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:05.809 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T09:06:05.852 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T09:06:05.852 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-13T09:06:05.901 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T09:06:05.901 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:06:05.902 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 882 Links: 1 2024-09-13T09:06:05.902 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:06:05.902 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-13 09:00:44.879716557 +0000 2024-09-13T09:06:05.902 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-13 09:00:44.623718845 +0000 2024-09-13T09:06:05.902 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-13 09:00:44.623718845 +0000 2024-09-13T09:06:05.902 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-09-13T09:06:05.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T09:06:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-13T09:06:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-13T09:06:05.954 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000454226 s, 1.1 MB/s 2024-09-13T09:06:05.956 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T09:06:06.005 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-13T09:06:06.052 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T09:06:06.052 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:06:06.052 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-09-13T09:06:06.052 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:06:06.052 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-13 09:00:45.111714485 +0000 2024-09-13T09:06:06.052 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-13 09:00:45.107714521 +0000 2024-09-13T09:06:06.052 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-13 09:00:45.107714521 +0000 2024-09-13T09:06:06.052 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-09-13T09:06:06.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T09:06:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-13T09:06:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-13T09:06:06.106 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000349724 s, 1.5 MB/s 2024-09-13T09:06:06.108 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T09:06:06.157 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-13T09:06:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T09:06:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:06:06.204 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 919 Links: 1 2024-09-13T09:06:06.204 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:06:06.204 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-13 09:00:45.827708090 +0000 2024-09-13T09:06:06.204 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-13 09:00:45.599710127 +0000 2024-09-13T09:06:06.205 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-13 09:00:45.599710127 +0000 2024-09-13T09:06:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-09-13T09:06:06.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T09:06:06.258 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-13T09:06:06.258 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-13T09:06:06.258 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000316568 s, 1.6 MB/s 2024-09-13T09:06:06.259 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T09:06:06.311 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-13T09:06:06.360 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T09:06:06.360 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:06:06.360 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-09-13T09:06:06.360 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:06:06.360 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-13 09:00:46.319703696 +0000 2024-09-13T09:06:06.360 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-13 09:00:46.067705947 +0000 2024-09-13T09:06:06.360 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-13 09:00:46.067705947 +0000 2024-09-13T09:06:06.360 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-09-13T09:06:06.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T09:06:06.415 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-13T09:06:06.415 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-13T09:06:06.415 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000389792 s, 1.3 MB/s 2024-09-13T09:06:06.416 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T09:06:06.465 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:06.466 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:06.466 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T09:06:06.472 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T09:06:06.472 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-09-13T09:06:06.521 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T09:06:06.521 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:06:06.522 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-09-13T09:06:06.522 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:06:06.522 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-13 09:00:45.794559858 +0000 2024-09-13T09:06:06.522 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-13 09:00:45.550562038 +0000 2024-09-13T09:06:06.522 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-13 09:00:45.550562038 +0000 2024-09-13T09:06:06.522 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2024-09-13T09:06:06.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T09:06:06.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-13T09:06:06.576 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-13T09:06:06.576 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000277896 s, 1.8 MB/s 2024-09-13T09:06:06.577 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T09:06:06.631 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-09-13T09:06:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T09:06:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:06:06.677 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-09-13T09:06:06.677 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:06:06.677 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-13 09:00:46.254555750 +0000 2024-09-13T09:06:06.677 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-13 09:00:46.026557785 +0000 2024-09-13T09:06:06.677 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-13 09:00:46.026557785 +0000 2024-09-13T09:06:06.677 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2024-09-13T09:06:06.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T09:06:06.731 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-13T09:06:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-13T09:06:06.732 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000254071 s, 2.0 MB/s 2024-09-13T09:06:06.733 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T09:06:06.782 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-09-13T09:06:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T09:06:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:06:06.829 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 912 Links: 1 2024-09-13T09:06:06.829 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:06:06.829 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-13 09:00:46.718551604 +0000 2024-09-13T09:06:06.829 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-13 09:00:46.474553785 +0000 2024-09-13T09:06:06.829 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-13 09:00:46.474553785 +0000 2024-09-13T09:06:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2024-09-13T09:06:06.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T09:06:06.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-13T09:06:06.883 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-13T09:06:06.883 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000303761 s, 1.7 MB/s 2024-09-13T09:06:06.885 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T09:06:06.938 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-09-13T09:06:06.986 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T09:06:06.986 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T09:06:06.986 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 936 Links: 1 2024-09-13T09:06:06.986 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T09:06:06.986 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-13 09:00:47.194547353 +0000 2024-09-13T09:06:06.986 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-13 09:00:46.926549746 +0000 2024-09-13T09:06:06.986 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-13 09:00:46.926549746 +0000 2024-09-13T09:06:06.986 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: - 2024-09-13T09:06:06.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T09:06:07.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-13T09:06:07.040 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-13T09:06:07.040 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000205094 s, 2.5 MB/s 2024-09-13T09:06:07.041 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T09:06:07.090 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:07.090 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T09:06:07.090 INFO:tasks.ceph:Generating config... 2024-09-13T09:06:07.091 INFO:tasks.ceph:[client] rbd cache = True 2024-09-13T09:06:07.091 INFO:tasks.ceph:[client] rbd cache policy = writeback 2024-09-13T09:06:07.092 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T09:06:07.092 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T09:06:07.092 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T09:06:07.092 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T09:06:07.092 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T09:06:07.092 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T09:06:07.092 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T09:06:07.092 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T09:06:07.092 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T09:06:07.092 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T09:06:07.092 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T09:06:07.092 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T09:06:07.092 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T09:06:07.093 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-13T09:06:07.094 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T09:06:07.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T09:06:07.129 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T09:06:07.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T09:06:07.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:06:07.272 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.c', '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]'), ('mon.b', '172.21.15.60')] 2024-09-13T09:06:07.273 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.60', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writeback'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-13T09:06:07.274 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:07.274 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T09:06:07.320 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --addv c '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]' --add b 172.21.15.60 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid fe16bf6d-4166-4417-80fd-1123617a33d3 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = pacific 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:fsid fe16bf6d-4166-4417-80fd-1123617a33d3 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-09-13T09:06:07.408192+0000 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-13T09:06:07.408192+0000 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 16 (pacific) 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-09-13T09:06:07.408 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-09-13T09:06:07.409 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.b 2024-09-13T09:06:07.409 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] mon.c 2024-09-13T09:06:07.409 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T09:06:07.410 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T09:06:07.415 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fe16bf6d-4166-4417-80fd-1123617a33d3... 2024-09-13T09:06:07.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:06:07.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T09:06:07.489 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-13T09:06:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-09-13T09:06:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:06:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-09-13T09:06:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-09-13T09:06:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.489 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-09-13T09:06:07.490 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:06:07.491 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-09-13T09:06:07.492 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.60" 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: mon client directed command retry = 5 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on pool no app = False 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: ms inject socket failures = 5000 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = fe16bf6d-4166-4417-80fd-1123617a33d3 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-09-13T09:06:07.493 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:06:07.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: bdev async discard = True 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: bdev enable discard = True 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore allocator = bitmap 2024-09-13T09:06:07.495 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: osd max markdown count = 1000 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-09-13T09:06:07.496 INFO:teuthology.orchestra.run.smithi031.stdout: osd shutdown pgref assert = True 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-13T09:06:07.497 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.498 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-09-13T09:06:07.499 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: rbd cache = True 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout: rbd cache policy = writeback 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-09-13T09:06:07.500 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-09-13T09:06:07.501 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: chdir = "" 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: auth supported = cephx 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: filestore xattr use omap = true 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: mon clock drift allowed = 1.000 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: osd crush chooseleaf type = 0 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: auth debug = true 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.502 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on old message = true 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: ms die on bug = true 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon pg warn max object skew = 0 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default size = 2 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow primary affinity = true 2024-09-13T09:06:07.503 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd allow pg remap = true 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on legacy crush tunables = false 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on crush straw calc version zero = false 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on no sortbitwise = false 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on osd down out interval zero = false 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on too few osds = false 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: mon_allow_pool_size_one = true 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: osd default data pool replay window = 5 2024-09-13T09:06:07.504 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: mon allow pool delete = true 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: mon cluster log file level = debug 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: debug asserts on shutdown = true 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: mon health detail to clog = false 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.60" 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: mon client directed command retry = 5 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: mon warn on pool no app = False 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: ms inject socket failures = 5000 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = fe16bf6d-4166-4417-80fd-1123617a33d3 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout:[osd] 2024-09-13T09:06:07.505 INFO:teuthology.orchestra.run.smithi060.stdout: osd journal size = 100 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub load threshold = 5.0 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd scrub max interval = 600 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd mclock profile = high_recovery_ops 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd recover clone overlap = true 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd recovery max chunk = 1048576 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug shutdown = true 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug op order = true 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug verify stray on activate = true 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd open classes on start = true 2024-09-13T09:06:07.506 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug pg log writeout = true 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: osd deep scrub update digest min age = 30 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: osd map max advance = 10 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: journal zero on create = true 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: filestore ondisk finisher threads = 3 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: filestore apply finisher threads = 3 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: bdev debug aio = true 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: osd debug misdirected ops = true 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: bdev async discard = True 2024-09-13T09:06:07.507 INFO:teuthology.orchestra.run.smithi060.stdout: bdev enable discard = True 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore allocator = bitmap 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore block size = 96636764160 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: bluestore fsck on mount = True 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluefs = 1/20 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: debug bluestore = 1/20 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: debug osd = 20 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: debug rocksdb = 4/10 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd full ratio = 0.9 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd nearfull ratio = 0.8 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: osd blocked scrub grace period = 3600 2024-09-13T09:06:07.508 INFO:teuthology.orchestra.run.smithi060.stdout: osd failsafe full ratio = 0.95 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: osd max markdown count = 1000 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: osd objectstore = bluestore 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: osd shutdown pgref assert = True 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr] 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: debug mgr = 20 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:06:07.509 INFO:teuthology.orchestra.run.smithi060.stdout: mgr/telemetry/nag = false 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout:[mon] 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: debug ms = 1 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: debug mon = 20 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: debug paxos = 20 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: debug auth = 20 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: mon data avail warn = 5 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: mon mgr mkfs grace = 240 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min pgs per osd = 4 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd reporter subtree level = osd 2024-09-13T09:06:07.510 INFO:teuthology.orchestra.run.smithi060.stdout: mon osd prime pg temp = true 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: mon reweight min bytes per osd = 10 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: # 1m isn't quite enough 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: mon_down_mkfs_grace = 2m 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: mon_warn_on_filestore_osds = false 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout:[client] 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: rgw cache enabled = true 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable ops log = true 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: rgw enable usage log = true 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T09:06:07.512 INFO:teuthology.orchestra.run.smithi060.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T09:06:07.513 INFO:teuthology.orchestra.run.smithi060.stdout: rbd cache = True 2024-09-13T09:06:07.513 INFO:teuthology.orchestra.run.smithi060.stdout: rbd cache policy = writeback 2024-09-13T09:06:07.513 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.a] 2024-09-13T09:06:07.513 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.c] 2024-09-13T09:06:07.513 INFO:teuthology.orchestra.run.smithi060.stdout:[mon.b] 2024-09-13T09:06:07.513 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T09:06:07.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T09:06:07.589 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T09:06:07.589 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:07.589 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T09:06:07.594 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:07.594 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T09:06:07.639 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:06:07.640 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:07.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:06:07.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:06:07.704 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:07.705 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:07.752 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:06:07.752 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:07.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T09:06:07.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T09:06:07.777 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:07.777 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:07.825 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T09:06:07.825 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T09:06:07.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T09:06:07.870 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T09:06:07.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T09:06:07.918 INFO:teuthology.orchestra.run.smithi060.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T09:06:07.940 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T09:06:07.940 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T09:06:07.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T09:06:07.976 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T09:06:07.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T09:06:08.020 INFO:teuthology.orchestra.run.smithi060.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T09:06:08.032 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T09:06:08.032 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi060.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T09:06:08.032 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T09:06:08.045 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:08.045 INFO:tasks.ceph:role: osd.0 2024-09-13T09:06:08.045 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:06:08.045 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T09:06:08.107 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:06:08.108 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:06:08.564 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-13T09:06:08.566 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-13T09:06:08.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T09:06:08.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T09:06:08.709 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:06:08.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:06:08.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T09:06:08.763 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:08.763 INFO:tasks.ceph:role: osd.1 2024-09-13T09:06:08.763 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:06:08.764 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T09:06:08.828 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:06:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:06:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:06:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:06:08.828 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:06:08.828 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:06:08.829 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:06:08.829 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:06:08.829 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:06:08.829 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:06:09.260 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-13T09:06:09.262 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-13T09:06:09.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T09:06:09.327 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T09:06:09.384 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:06:09.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:06:09.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T09:06:09.439 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:09.439 INFO:tasks.ceph:role: osd.2 2024-09-13T09:06:09.439 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:06:09.439 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T09:06:09.509 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:06:09.509 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:06:09.509 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:06:09.509 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:06:09.509 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:06:09.510 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:06:09.510 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:06:09.510 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:06:09.510 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:06:09.510 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:06:09.965 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-13T09:06:09.968 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-13T09:06:09.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T09:06:10.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T09:06:10.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:06:10.075 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:06:10.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T09:06:10.129 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:10.129 INFO:tasks.ceph:role: osd.3 2024-09-13T09:06:10.129 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:06:10.129 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T09:06:10.193 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:06:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:06:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:06:10.193 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:06:10.194 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:06:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:06:10.194 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:06:10.194 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:06:10.194 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:06:10.194 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:06:10.648 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-13T09:06:10.650 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-13T09:06:10.651 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T09:06:10.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T09:06:10.806 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:06:10.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:06:10.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:10.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:10.925+0000 7f5f8ef837c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T09:06:10.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:10.929+0000 7f5f8ef837c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T09:06:10.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:10.929+0000 7f5f8ef837c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:06:10.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:10.929+0000 7f5f8ef837c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:06:10.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:10.929+0000 7f5f8ef837c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T09:06:10.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:10.937+0000 7f5f8ef837c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T09:06:13.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T09:06:13.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:14.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:14.013+0000 7f3a3292f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T09:06:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:14.013+0000 7f3a3292f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T09:06:14.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:14.013+0000 7f3a3292f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:06:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:14.013+0000 7f3a3292f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:06:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:14.013+0000 7f3a3292f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T09:06:14.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:14.013+0000 7f3a3292f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T09:06:16.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T09:06:17.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:17.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:17.129+0000 7fd7d1e947c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T09:06:17.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:17.129+0000 7fd7d1e947c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T09:06:17.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:17.133+0000 7fd7d1e947c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:06:17.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:17.133+0000 7fd7d1e947c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:06:17.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:17.133+0000 7fd7d1e947c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T09:06:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:17.133+0000 7fd7d1e947c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T09:06:20.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T09:06:20.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:20.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:20.257+0000 7efef758c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T09:06:20.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:20.257+0000 7efef758c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T09:06:20.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:20.257+0000 7efef758c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:06:20.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:20.257+0000 7efef758c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:06:20.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:20.257+0000 7efef758c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T09:06:20.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:06:20.261+0000 7efef758c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T09:06:23.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T09:06:23.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T09:06:23.304 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:23.304 INFO:tasks.ceph:role: osd.4 2024-09-13T09:06:23.305 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:06:23.305 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:06:23.361 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:06:23.384 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-13T09:06:23.386 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-13T09:06:23.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T09:06:23.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T09:06:23.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:06:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:06:23.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T09:06:23.547 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:23.547 INFO:tasks.ceph:role: osd.5 2024-09-13T09:06:23.547 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:06:23.548 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T09:06:23.600 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:06:23.600 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:06:23.600 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:06:23.601 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:06:23.601 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:06:23.601 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:06:23.601 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:06:23.601 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:06:23.601 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:06:23.601 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:06:23.625 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-13T09:06:23.627 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-13T09:06:23.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T09:06:23.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T09:06:23.737 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:06:23.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:06:23.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T09:06:23.791 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:23.791 INFO:tasks.ceph:role: osd.6 2024-09-13T09:06:23.791 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:06:23.791 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:06:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:06:23.845 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:06:23.867 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-13T09:06:23.869 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-13T09:06:23.870 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T09:06:23.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T09:06:23.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:06:23.982 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:06:23.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T09:06:24.037 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T09:06:24.037 INFO:tasks.ceph:role: osd.7 2024-09-13T09:06:24.037 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:06:24.037 DEBUG:teuthology.orchestra.run.smithi060:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T09:06:24.092 INFO:teuthology.orchestra.run.smithi060.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T09:06:24.092 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T09:06:24.093 INFO:teuthology.orchestra.run.smithi060.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T09:06:24.093 INFO:teuthology.orchestra.run.smithi060.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T09:06:24.093 INFO:teuthology.orchestra.run.smithi060.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T09:06:24.093 INFO:teuthology.orchestra.run.smithi060.stdout: = sunit=0 swidth=0 blks 2024-09-13T09:06:24.093 INFO:teuthology.orchestra.run.smithi060.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T09:06:24.093 INFO:teuthology.orchestra.run.smithi060.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T09:06:24.093 INFO:teuthology.orchestra.run.smithi060.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T09:06:24.093 INFO:teuthology.orchestra.run.smithi060.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T09:06:24.116 INFO:teuthology.orchestra.run.smithi060.stdout:Discarding blocks...Done. 2024-09-13T09:06:24.118 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi060.front.sepia.ceph.com -o noatime 2024-09-13T09:06:24.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T09:06:24.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T09:06:24.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:06:24.230 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T09:06:24.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:24.345 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:24.342+0000 7ffb820187c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T09:06:24.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:24.342+0000 7ffb820187c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T09:06:24.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:24.342+0000 7ffb820187c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:06:24.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:24.342+0000 7ffb820187c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:06:24.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:24.342+0000 7ffb820187c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T09:06:24.346 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:24.342+0000 7ffb820187c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T09:06:27.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T09:06:27.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:27.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:27.410+0000 7fe8997867c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T09:06:27.416 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:27.410+0000 7fe8997867c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T09:06:27.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:27.410+0000 7fe8997867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:06:27.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:27.414+0000 7fe8997867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:06:27.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:27.414+0000 7fe8997867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T09:06:27.417 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:27.414+0000 7fe8997867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T09:06:30.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T09:06:30.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:30.533 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:30.530+0000 7fe05f8f37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T09:06:30.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:30.530+0000 7fe05f8f37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T09:06:30.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:30.530+0000 7fe05f8f37c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:06:30.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:30.530+0000 7fe05f8f37c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:06:30.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:30.530+0000 7fe05f8f37c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T09:06:30.534 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:30.530+0000 7fe05f8f37c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T09:06:33.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T09:06:33.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:33.622+0000 7f0194be27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T09:06:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:33.622+0000 7f0194be27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T09:06:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:33.622+0000 7f0194be27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:06:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:33.622+0000 7f0194be27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:06:33.628 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:33.622+0000 7f0194be27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T09:06:33.629 INFO:teuthology.orchestra.run.smithi060.stderr:2024-09-13T09:06:33.626+0000 7f0194be27c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T09:06:36.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T09:06:36.667 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T09:06:36.667 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:36.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T09:06:36.683 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:36.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T09:06:36.740 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:36.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T09:06:36.796 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:36.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T09:06:36.851 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:36.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T09:06:36.907 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:36.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T09:06:36.921 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:36.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T09:06:36.976 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:36.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T09:06:37.036 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:37.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T09:06:37.093 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:37.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T09:06:37.148 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:37.148 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T09:06:37.154 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:37.154 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T09:06:37.197 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T09:06:37.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T09:06:37.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T09:06:37.242 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB/AORm8qfjMxAAy7uiN81nxgARpXSgRQxb4A== 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCCAORmjXptNxAA+zEAAnKXkotRTRjg/a7ejA== 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCGAORmvUnzABAAXZKuxfVnyOU+ZKWWItXICA== 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCJAORmdZQLCBAAM0Io5oMck+sTACMLcscsNg== 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCMAORmNrKpDxAA90POB4HVzxZL9nTSqxssdg== 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-09-13T09:06:37.243 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB/AORmH9e2NhAAatsJUSWspxuWR+OBVuqg3w== 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCQAORmBJWXFBAAw40ujjnSZQo9ECxeO4ImCw== 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCTAORmkyrMGBAAsL0/qQkinUpY6hvYdJTv3g== 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCWAORm/VbMHxAADeOP0kuDPlMyls4qpft3+A== 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCZAORmgUZrJRAAbVIhyIRf7hmbEhl1oyI5yw== 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQB/AORmXvgwOhAAk/PrKQD+HJ7sQrVfw5R4Pw== 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2024-09-13T09:06:37.244 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCAAORmX6Y8ARAAAi6QODAVAPS+R+OojqhlNg== 2024-09-13T09:06:37.251 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.y] 2024-09-13T09:06:37.251 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQB/AORm8qfjMxAAy7uiN81nxgARpXSgRQxb4A== 2024-09-13T09:06:37.251 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.0] 2024-09-13T09:06:37.251 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCCAORmjXptNxAA+zEAAnKXkotRTRjg/a7ejA== 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.1] 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCGAORmvUnzABAAXZKuxfVnyOU+ZKWWItXICA== 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.2] 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCJAORmdZQLCBAAM0Io5oMck+sTACMLcscsNg== 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.3] 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCMAORmNrKpDxAA90POB4HVzxZL9nTSqxssdg== 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout:[mgr.x] 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQB/AORmH9e2NhAAatsJUSWspxuWR+OBVuqg3w== 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.4] 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCQAORmBJWXFBAAw40ujjnSZQo9ECxeO4ImCw== 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.5] 2024-09-13T09:06:37.252 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCTAORmkyrMGBAAsL0/qQkinUpY6hvYdJTv3g== 2024-09-13T09:06:37.253 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.6] 2024-09-13T09:06:37.253 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCWAORm/VbMHxAADeOP0kuDPlMyls4qpft3+A== 2024-09-13T09:06:37.253 INFO:teuthology.orchestra.run.smithi060.stdout:[osd.7] 2024-09-13T09:06:37.253 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCZAORmgUZrJRAAbVIhyIRf7hmbEhl1oyI5yw== 2024-09-13T09:06:37.253 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-13T09:06:37.253 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQB/AORmXvgwOhAAk/PrKQD+HJ7sQrVfw5R4Pw== 2024-09-13T09:06:37.253 INFO:teuthology.orchestra.run.smithi060.stdout:[client.1] 2024-09-13T09:06:37.253 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQCAAORmX6Y8ARAAAi6QODAVAPS+R+OojqhlNg== 2024-09-13T09:06:37.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:06:37.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:06:37.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:06:37.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T09:06:37.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:37.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:38.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:38.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T09:06:38.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:06:38.128 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:06:38.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:06:38.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T09:06:38.262 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T09:06:38.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T09:06:38.316 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:06:38.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T09:06:38.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T09:06:38.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:06:38.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T09:06:38.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T09:06:38.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T09:06:38.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T09:06:38.696 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:38.698 DEBUG:teuthology.orchestra.run.smithi060:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T09:06:38.744 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T09:06:38.744 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T09:06:38.744 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T09:06:38.747 INFO:tasks.ceph.mon.a:Started 2024-09-13T09:06:38.747 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T09:06:38.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T09:06:38.749 INFO:tasks.ceph.mon.c:Started 2024-09-13T09:06:38.750 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T09:06:38.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T09:06:38.794 INFO:tasks.ceph.mon.b:Started 2024-09-13T09:06:38.794 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T09:06:38.794 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T09:06:38.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T09:06:38.796 INFO:tasks.ceph.mgr.y:Started 2024-09-13T09:06:38.797 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T09:06:38.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T09:06:38.799 INFO:tasks.ceph.mgr.x:Started 2024-09-13T09:06:38.799 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T09:06:38.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T09:06:44.284 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-13T09:06:44.317 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T09:06:44.317 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T09:06:44.317 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:44.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T09:06:44.363 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:44.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T09:06:44.389 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:44.386+0000 7f83c2cf5180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T09:06:44.422 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:44.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T09:06:44.488 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-13T09:06:44.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T09:06:44.571 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:44.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T09:06:44.602 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:44.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T09:06:44.663 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:44.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T09:06:44.729 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-13T09:06:44.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T09:06:44.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd new 33c6fc7d-9828-4500-b103-b96d1cd7f420 0 2024-09-13T09:06:44.944 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:44.938+0000 7f83c2cf5180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T09:06:45.056 INFO:teuthology.orchestra.run.smithi060.stdout:0 2024-09-13T09:06:45.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd new 869ea3c5-8e6e-4b49-b70e-083f242a7aae 1 2024-09-13T09:06:45.149 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:45.146+0000 7f83c2cf5180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T09:06:45.159 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:45.157+0000 7fd4f2648180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T09:06:45.287 INFO:teuthology.orchestra.run.smithi060.stdout:1 2024-09-13T09:06:45.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd new e144d496-76ee-46e0-bdc6-135c77128535 2 2024-09-13T09:06:45.504 INFO:teuthology.orchestra.run.smithi060.stdout:2 2024-09-13T09:06:45.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd new 696b9414-9c19-4984-a8ff-14548e46b357 3 2024-09-13T09:06:45.695 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:45.689+0000 7fd4f2648180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T09:06:45.721 INFO:teuthology.orchestra.run.smithi060.stdout:3 2024-09-13T09:06:45.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd new ac58040d-9b54-42ca-b143-1816abb54816 4 2024-09-13T09:06:45.738 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:45.734+0000 7f83c2cf5180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T09:06:45.795 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:45.790+0000 7f83c2cf5180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T09:06:45.851 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:45.846+0000 7f83c2cf5180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T09:06:45.918 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:45.913+0000 7fd4f2648180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T09:06:45.934 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:45.930+0000 7f83c2cf5180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T09:06:45.938 INFO:teuthology.orchestra.run.smithi060.stdout:4 2024-09-13T09:06:45.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd new e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2 5 2024-09-13T09:06:45.991 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:45.986+0000 7f83c2cf5180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T09:06:46.073 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:46.070+0000 7f83c2cf5180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T09:06:46.129 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:46.126+0000 7f83c2cf5180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T09:06:46.161 INFO:teuthology.orchestra.run.smithi060.stdout:5 2024-09-13T09:06:46.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd new d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d 6 2024-09-13T09:06:46.291 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:46.286+0000 7f83c2cf5180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T09:06:46.354 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:46.350+0000 7f83c2cf5180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T09:06:46.426 INFO:teuthology.orchestra.run.smithi060.stdout:6 2024-09-13T09:06:46.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo ceph --cluster ceph osd new f36c332c-9497-47e8-b943-f6b6dcaebeb8 7 2024-09-13T09:06:46.512 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:46.509+0000 7fd4f2648180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T09:06:46.568 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:46.565+0000 7fd4f2648180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T09:06:46.623 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:46.621+0000 7fd4f2648180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T09:06:46.641 INFO:teuthology.orchestra.run.smithi060.stdout:7 2024-09-13T09:06:46.658 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T09:06:46.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T09:06:46.661 INFO:tasks.ceph.osd.0:Started 2024-09-13T09:06:46.661 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T09:06:46.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T09:06:46.663 INFO:tasks.ceph.osd.1:Started 2024-09-13T09:06:46.664 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T09:06:46.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T09:06:46.666 INFO:tasks.ceph.osd.2:Started 2024-09-13T09:06:46.666 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T09:06:46.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T09:06:46.670 INFO:tasks.ceph.osd.3:Started 2024-09-13T09:06:46.670 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T09:06:46.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T09:06:46.672 INFO:tasks.ceph.osd.4:Started 2024-09-13T09:06:46.673 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T09:06:46.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T09:06:46.675 INFO:tasks.ceph.osd.5:Started 2024-09-13T09:06:46.675 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T09:06:46.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T09:06:46.677 INFO:tasks.ceph.osd.6:Started 2024-09-13T09:06:46.678 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T09:06:46.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T09:06:46.680 INFO:tasks.ceph.osd.7:Started 2024-09-13T09:06:46.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:06:46.729 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:46.725+0000 7fd4f2648180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T09:06:46.790 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:46.785+0000 7fd4f2648180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T09:06:46.873 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:46.869+0000 7fd4f2648180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T09:06:46.875 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:06:46.875 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:06:46.637496+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:06:46.637496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:06:46.891 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T09:06:46.891 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T09:06:46.928 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:46.925+0000 7fd4f2648180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T09:06:47.018 INFO:tasks.ceph.mgr.x.smithi060.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T09:06:47.018 INFO:tasks.ceph.mgr.x.smithi060.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T09:06:47.019 INFO:tasks.ceph.mgr.x.smithi060.stderr: from numpy import show_config as show_numpy_config 2024-09-13T09:06:47.086 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:47.081+0000 7fd4f2648180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T09:06:47.096 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:47.090+0000 7f83c2cf5180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T09:06:47.149 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:47.145+0000 7fd4f2648180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T09:06:47.281 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:47.278+0000 7f83c2cf5180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:06:47.296 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:06:47.293+0000 7fab7aec87c0 -1 Falling back to public interface 2024-09-13T09:06:47.303 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:06:47.301+0000 7f0f4686f7c0 -1 Falling back to public interface 2024-09-13T09:06:47.319 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:06:47.317+0000 7f7378f9a7c0 -1 Falling back to public interface 2024-09-13T09:06:47.320 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:06:47.317+0000 7efe542d57c0 -1 Falling back to public interface 2024-09-13T09:06:47.330 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:06:47.326+0000 7f6a685687c0 -1 Falling back to public interface 2024-09-13T09:06:47.345 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:06:47.342+0000 7f3f616857c0 -1 Falling back to public interface 2024-09-13T09:06:47.346 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:06:47.342+0000 7f4ea070a7c0 -1 Falling back to public interface 2024-09-13T09:06:47.349 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:06:47.346+0000 7fb1d58ea7c0 -1 Falling back to public interface 2024-09-13T09:06:47.446 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:47.442+0000 7f83c2cf5180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T09:06:47.511 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:47.506+0000 7f83c2cf5180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T09:06:47.651 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:47.646+0000 7f83c2cf5180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:06:47.695 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:06:47.787 INFO:tasks.ceph.mgr.y.smithi031.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T09:06:47.787 INFO:tasks.ceph.mgr.y.smithi031.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T09:06:47.787 INFO:tasks.ceph.mgr.y.smithi031.stderr: from numpy import show_config as show_numpy_config 2024-09-13T09:06:47.825 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:47.821+0000 7fd4f2648180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T09:06:47.885 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-13T09:06:47.885 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:06:46.637496+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T09:06:46.637496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:06:47.901 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T09:06:47.974 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:47.969+0000 7fd4f2648180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:06:48.155 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:48.153+0000 7fd4f2648180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T09:06:48.225 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:48.221+0000 7fd4f2648180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T09:06:48.299 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:48.294+0000 7f83c2cf5180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T09:06:48.363 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:48.361+0000 7fd4f2648180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T09:06:48.378 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:48.374+0000 7f83c2cf5180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T09:06:48.567 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:48.562+0000 7f83c2cf5180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T09:06:48.645 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:48.642+0000 7f83c2cf5180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T09:06:48.947 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:48.942+0000 7f83c2cf5180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T09:06:48.983 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:48.981+0000 7fd4f2648180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T09:06:49.011 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:49.006+0000 7f83c2cf5180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T09:06:49.064 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:49.061+0000 7fd4f2648180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T09:06:49.257 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:49.253+0000 7fd4f2648180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T09:06:49.333 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:49.329+0000 7fd4f2648180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T09:06:49.635 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:49.629+0000 7fd4f2648180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T09:06:49.699 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-09-13T09:06:49.697+0000 7fd4f2648180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T09:06:50.051 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:50.046+0000 7f836a2aa640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T09:06:50.057 INFO:tasks.ceph.mgr.x.smithi060.stderr:2024-09-13T09:06:50.054+0000 7f836a2aa640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T09:06:51.145 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:06:51.142+0000 7f3f616857c0 -1 osd.5 0 log_to_monitors true 2024-09-13T09:06:51.198 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:06:51.193+0000 7f7378f9a7c0 -1 osd.0 0 log_to_monitors true 2024-09-13T09:06:51.206 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:06:51.201+0000 7efe542d57c0 -1 osd.2 0 log_to_monitors true 2024-09-13T09:06:51.207 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:06:51.202+0000 7fb1d58ea7c0 -1 osd.6 0 log_to_monitors true 2024-09-13T09:06:51.278 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:06:51.273+0000 7f0f4686f7c0 -1 osd.3 0 log_to_monitors true 2024-09-13T09:06:51.287 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:06:51.285+0000 7fab7aec87c0 -1 osd.1 0 log_to_monitors true 2024-09-13T09:06:51.300 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:06:51.298+0000 7f4ea070a7c0 -1 osd.7 0 log_to_monitors true 2024-09-13T09:06:51.318 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:06:51.314+0000 7f6a685687c0 -1 osd.4 0 log_to_monitors true 2024-09-13T09:06:53.058 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:06:53.054+0000 7f3f5ce07640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T09:06:53.093 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:06:53.090+0000 7f3f5544c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:06:54.471 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:06:54.466+0000 7f6a65525640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T09:06:54.478 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:06:54.474+0000 7fb1d106c640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T09:06:54.497 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:06:54.494+0000 7f6a5c32f640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:06:54.509 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:06:54.506+0000 7fb1c96b1640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:06:54.517 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:06:54.514+0000 7f4e9be8c640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T09:06:54.558 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:06:54.554+0000 7f4e944d1640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:06:54.708 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:06:55.237 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-13T09:06:55.237 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":14,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:06:54.553271+0000","last_up_change":"2024-09-13T09:06:54.553271+0000","last_in_change":"2024-09-13T09:06:46.637496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6801","nonce":3778074750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6803","nonce":3778074750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6807","nonce":3778074750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6805","nonce":3778074750}]},"public_addr":"172.21.15.60:6801/3778074750","cluster_addr":"172.21.15.60:6803/3778074750","heartbeat_back_addr":"172.21.15.60:6807/3778074750","heartbeat_front_addr":"172.21.15.60:6805/3778074750","state":["exists","up"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6809","nonce":1926603087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6813","nonce":1926603087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6824","nonce":1926603087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6817","nonce":1926603087}]},"public_addr":"172.21.15.60:6809/1926603087","cluster_addr":"172.21.15.60:6813/1926603087","heartbeat_back_addr":"172.21.15.60:6824/1926603087","heartbeat_front_addr":"172.21.15.60:6817/1926603087","state":["exists","up"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6821","nonce":766305500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6825","nonce":766305500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6831","nonce":766305500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6828","nonce":766305500}]},"public_addr":"172.21.15.60:6821/766305500","cluster_addr":"172.21.15.60:6825/766305500","heartbeat_back_addr":"172.21.15.60:6831/766305500","heartbeat_front_addr":"172.21.15.60:6828/766305500","state":["exists","up"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.158089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:06:55.270 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-13T09:06:55.493 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:06:55.489+0000 7fab77e85640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T09:06:55.505 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:06:55.501+0000 7efe4fa57640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T09:06:55.521 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:06:55.517+0000 7f737471c640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T09:06:55.523 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:06:55.521+0000 7fab6ec8f640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:06:55.530 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:06:55.525+0000 7f0f41ff1640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T09:06:55.535 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:06:55.533+0000 7efe4809c640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:06:55.543 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:06:55.541+0000 7f736cd61640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:06:55.551 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:06:55.549+0000 7f0f3a636640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:07:02.077 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T09:07:02.380 INFO:teuthology.misc.health.smithi031.stdout: 2024-09-13T09:07:02.380 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":17,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:06:57.617177+0000","last_up_change":"2024-09-13T09:06:55.611487+0000","last_in_change":"2024-09-13T09:06:46.637496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:06:55.131330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6818","nonce":3291756747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6822","nonce":3291756747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6830","nonce":3291756747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6826","nonce":3291756747}]},"public_addr":"172.21.15.31:6818/3291756747","cluster_addr":"172.21.15.31:6822/3291756747","heartbeat_back_addr":"172.21.15.31:6830/3291756747","heartbeat_front_addr":"172.21.15.31:6826/3291756747","state":["exists","up"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6801","nonce":533520566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6803","nonce":533520566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6807","nonce":533520566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6805","nonce":533520566}]},"public_addr":"172.21.15.31:6801/533520566","cluster_addr":"172.21.15.31:6803/533520566","heartbeat_back_addr":"172.21.15.31:6807/533520566","heartbeat_front_addr":"172.21.15.31:6805/533520566","state":["exists","up"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6819","nonce":2253604482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6823","nonce":2253604482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6829","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6831","nonce":2253604482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6825","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6827","nonce":2253604482}]},"public_addr":"172.21.15.31:6819/2253604482","cluster_addr":"172.21.15.31:6823/2253604482","heartbeat_back_addr":"172.21.15.31:6831/2253604482","heartbeat_front_addr":"172.21.15.31:6827/2253604482","state":["exists","up"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6809","nonce":3367214693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6811","nonce":3367214693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6815","nonce":3367214693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6813","nonce":3367214693}]},"public_addr":"172.21.15.31:6809/3367214693","cluster_addr":"172.21.15.31:6811/3367214693","heartbeat_back_addr":"172.21.15.31:6815/3367214693","heartbeat_front_addr":"172.21.15.31:6813/3367214693","state":["exists","up"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6801","nonce":3778074750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6803","nonce":3778074750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6807","nonce":3778074750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6805","nonce":3778074750}]},"public_addr":"172.21.15.60:6801/3778074750","cluster_addr":"172.21.15.60:6803/3778074750","heartbeat_back_addr":"172.21.15.60:6807/3778074750","heartbeat_front_addr":"172.21.15.60:6805/3778074750","state":["exists","up"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6809","nonce":1926603087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6813","nonce":1926603087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6824","nonce":1926603087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6817","nonce":1926603087}]},"public_addr":"172.21.15.60:6809/1926603087","cluster_addr":"172.21.15.60:6813/1926603087","heartbeat_back_addr":"172.21.15.60:6824/1926603087","heartbeat_front_addr":"172.21.15.60:6817/1926603087","state":["exists","up"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6821","nonce":766305500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6825","nonce":766305500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6831","nonce":766305500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6828","nonce":766305500}]},"public_addr":"172.21.15.60:6821/766305500","cluster_addr":"172.21.15.60:6825/766305500","heartbeat_back_addr":"172.21.15.60:6831/766305500","heartbeat_front_addr":"172.21.15.60:6828/766305500","state":["exists","up"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6812","nonce":1032956039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6816","nonce":1032956039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6829","nonce":1032956039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6822","nonce":1032956039}]},"public_addr":"172.21.15.60:6812/1032956039","cluster_addr":"172.21.15.60:6816/1032956039","heartbeat_back_addr":"172.21.15.60:6829/1032956039","heartbeat_front_addr":"172.21.15.60:6822/1032956039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.183436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.264080+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.253587+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.244429+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.369458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.158089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.217295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.332254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:07:02.398 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T09:07:02.398 INFO:tasks.ceph:Creating RBD pool 2024-09-13T09:07:02.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T09:07:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2024-09-13T09:07:03.141 DEBUG:teuthology.orchestra.run.smithi031:> rbd --cluster ceph pool init rbd 2024-09-13T09:07:06.171 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T09:07:06.171 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T09:07:06.171 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T09:07:06.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T09:07:06.172 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T09:07:06.525 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:06.543 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":511010661},{"type":"v1","addr":"172.21.15.60:6833","nonce":511010661}]},"active_addr":"172.21.15.60:6833/511010661","active_change":"2024-09-13T09:06:49.032505+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":1339976700}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2157845931}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2123330464}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":365871594}]}]} 2024-09-13T09:07:06.547 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T09:07:06.547 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T09:07:06.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:07:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:06.852 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:07:06.128047+0000","last_up_change":"2024-09-13T09:06:55.611487+0000","last_in_change":"2024-09-13T09:06:46.637496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:06:55.131330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:07:02.743102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6818","nonce":3291756747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6822","nonce":3291756747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6830","nonce":3291756747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6826","nonce":3291756747}]},"public_addr":"172.21.15.31:6818/3291756747","cluster_addr":"172.21.15.31:6822/3291756747","heartbeat_back_addr":"172.21.15.31:6830/3291756747","heartbeat_front_addr":"172.21.15.31:6826/3291756747","state":["exists","up"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6801","nonce":533520566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6803","nonce":533520566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6807","nonce":533520566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6805","nonce":533520566}]},"public_addr":"172.21.15.31:6801/533520566","cluster_addr":"172.21.15.31:6803/533520566","heartbeat_back_addr":"172.21.15.31:6807/533520566","heartbeat_front_addr":"172.21.15.31:6805/533520566","state":["exists","up"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6819","nonce":2253604482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6823","nonce":2253604482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6829","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6831","nonce":2253604482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6825","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6827","nonce":2253604482}]},"public_addr":"172.21.15.31:6819/2253604482","cluster_addr":"172.21.15.31:6823/2253604482","heartbeat_back_addr":"172.21.15.31:6831/2253604482","heartbeat_front_addr":"172.21.15.31:6827/2253604482","state":["exists","up"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6809","nonce":3367214693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6811","nonce":3367214693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6815","nonce":3367214693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6813","nonce":3367214693}]},"public_addr":"172.21.15.31:6809/3367214693","cluster_addr":"172.21.15.31:6811/3367214693","heartbeat_back_addr":"172.21.15.31:6815/3367214693","heartbeat_front_addr":"172.21.15.31:6813/3367214693","state":["exists","up"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6801","nonce":3778074750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6803","nonce":3778074750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6807","nonce":3778074750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6805","nonce":3778074750}]},"public_addr":"172.21.15.60:6801/3778074750","cluster_addr":"172.21.15.60:6803/3778074750","heartbeat_back_addr":"172.21.15.60:6807/3778074750","heartbeat_front_addr":"172.21.15.60:6805/3778074750","state":["exists","up"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6809","nonce":1926603087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6813","nonce":1926603087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6824","nonce":1926603087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6817","nonce":1926603087}]},"public_addr":"172.21.15.60:6809/1926603087","cluster_addr":"172.21.15.60:6813/1926603087","heartbeat_back_addr":"172.21.15.60:6824/1926603087","heartbeat_front_addr":"172.21.15.60:6817/1926603087","state":["exists","up"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6821","nonce":766305500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6825","nonce":766305500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6831","nonce":766305500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6828","nonce":766305500}]},"public_addr":"172.21.15.60:6821/766305500","cluster_addr":"172.21.15.60:6825/766305500","heartbeat_back_addr":"172.21.15.60:6831/766305500","heartbeat_front_addr":"172.21.15.60:6828/766305500","state":["exists","up"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6812","nonce":1032956039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6816","nonce":1032956039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6829","nonce":1032956039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6822","nonce":1032956039}]},"public_addr":"172.21.15.60:6812/1032956039","cluster_addr":"172.21.15.60:6816/1032956039","heartbeat_back_addr":"172.21.15.60:6829/1032956039","heartbeat_front_addr":"172.21.15.60:6822/1032956039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.183436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.264080+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.253587+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.244429+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.369458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.158089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.217295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.332254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:07:06.871 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T09:07:06.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:07:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:07.174 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:07:06.128047+0000","last_up_change":"2024-09-13T09:06:55.611487+0000","last_in_change":"2024-09-13T09:06:46.637496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:06:55.131330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:07:02.743102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6818","nonce":3291756747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6822","nonce":3291756747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6830","nonce":3291756747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6826","nonce":3291756747}]},"public_addr":"172.21.15.31:6818/3291756747","cluster_addr":"172.21.15.31:6822/3291756747","heartbeat_back_addr":"172.21.15.31:6830/3291756747","heartbeat_front_addr":"172.21.15.31:6826/3291756747","state":["exists","up"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6801","nonce":533520566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6803","nonce":533520566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6807","nonce":533520566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6805","nonce":533520566}]},"public_addr":"172.21.15.31:6801/533520566","cluster_addr":"172.21.15.31:6803/533520566","heartbeat_back_addr":"172.21.15.31:6807/533520566","heartbeat_front_addr":"172.21.15.31:6805/533520566","state":["exists","up"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6819","nonce":2253604482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6823","nonce":2253604482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6829","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6831","nonce":2253604482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6825","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6827","nonce":2253604482}]},"public_addr":"172.21.15.31:6819/2253604482","cluster_addr":"172.21.15.31:6823/2253604482","heartbeat_back_addr":"172.21.15.31:6831/2253604482","heartbeat_front_addr":"172.21.15.31:6827/2253604482","state":["exists","up"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6809","nonce":3367214693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6811","nonce":3367214693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6815","nonce":3367214693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6813","nonce":3367214693}]},"public_addr":"172.21.15.31:6809/3367214693","cluster_addr":"172.21.15.31:6811/3367214693","heartbeat_back_addr":"172.21.15.31:6815/3367214693","heartbeat_front_addr":"172.21.15.31:6813/3367214693","state":["exists","up"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6801","nonce":3778074750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6803","nonce":3778074750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6807","nonce":3778074750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6805","nonce":3778074750}]},"public_addr":"172.21.15.60:6801/3778074750","cluster_addr":"172.21.15.60:6803/3778074750","heartbeat_back_addr":"172.21.15.60:6807/3778074750","heartbeat_front_addr":"172.21.15.60:6805/3778074750","state":["exists","up"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6809","nonce":1926603087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6813","nonce":1926603087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6824","nonce":1926603087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6817","nonce":1926603087}]},"public_addr":"172.21.15.60:6809/1926603087","cluster_addr":"172.21.15.60:6813/1926603087","heartbeat_back_addr":"172.21.15.60:6824/1926603087","heartbeat_front_addr":"172.21.15.60:6817/1926603087","state":["exists","up"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6821","nonce":766305500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6825","nonce":766305500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6831","nonce":766305500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6828","nonce":766305500}]},"public_addr":"172.21.15.60:6821/766305500","cluster_addr":"172.21.15.60:6825/766305500","heartbeat_back_addr":"172.21.15.60:6831/766305500","heartbeat_front_addr":"172.21.15.60:6828/766305500","state":["exists","up"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6812","nonce":1032956039}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6816","nonce":1032956039}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6829","nonce":1032956039}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":1032956039},{"type":"v1","addr":"172.21.15.60:6822","nonce":1032956039}]},"public_addr":"172.21.15.60:6812/1032956039","cluster_addr":"172.21.15.60:6816/1032956039","heartbeat_back_addr":"172.21.15.60:6829/1032956039","heartbeat_front_addr":"172.21.15.60:6822/1032956039","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.183436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.264080+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.253587+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.244429+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.369458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.158089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.217295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.332254+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:07:07.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T09:07:07.192 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T09:07:07.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T09:07:07.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T09:07:07.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T09:07:07.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T09:07:07.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T09:07:07.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T09:07:07.426 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-13T09:07:07.426 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:07:07.460 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-13T09:07:07.460 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:07:07.461 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-13T09:07:07.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:07:07.463 INFO:teuthology.orchestra.run.smithi031.stdout:60129542149 2024-09-13T09:07:07.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:07:07.464 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-13T09:07:07.465 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:07:07.469 INFO:teuthology.orchestra.run.smithi031.stdout:60129542148 2024-09-13T09:07:07.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:07:07.478 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-13T09:07:07.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:07:07.495 INFO:teuthology.orchestra.run.smithi031.stdout:55834574853 2024-09-13T09:07:07.495 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:07:07.946 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-13T09:07:07.955 INFO:teuthology.orchestra.run.smithi031.stdout:60129542148 2024-09-13T09:07:07.972 INFO:teuthology.orchestra.run.smithi031.stdout:64424509443 2024-09-13T09:07:07.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.3 2024-09-13T09:07:07.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-09-13T09:07:07.984 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-13T09:07:07.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.2 2024-09-13T09:07:08.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2024-09-13T09:07:08.012 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-13T09:07:08.025 INFO:teuthology.orchestra.run.smithi031.stdout:55834574852 2024-09-13T09:07:08.032 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-13T09:07:08.036 INFO:teuthology.orchestra.run.smithi031.stdout:60129542147 2024-09-13T09:07:08.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.7 2024-09-13T09:07:08.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-09-13T09:07:08.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2024-09-13T09:07:08.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-09-13T09:07:08.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:07:08.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:07:09.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:07:09.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:07:09.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:07:09.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:07:09.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:07:09.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:07:09.507 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-13T09:07:09.527 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-13T09:07:09.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.3 2024-09-13T09:07:09.532 DEBUG:teuthology.parallel:result is None 2024-09-13T09:07:09.534 INFO:teuthology.orchestra.run.smithi031.stdout:64424509444 2024-09-13T09:07:09.539 INFO:teuthology.orchestra.run.smithi031.stdout:60129542149 2024-09-13T09:07:09.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-09-13T09:07:09.551 DEBUG:teuthology.parallel:result is None 2024-09-13T09:07:09.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2024-09-13T09:07:09.564 DEBUG:teuthology.parallel:result is None 2024-09-13T09:07:09.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.2 2024-09-13T09:07:09.566 DEBUG:teuthology.parallel:result is None 2024-09-13T09:07:09.578 INFO:teuthology.orchestra.run.smithi031.stdout:60129542148 2024-09-13T09:07:09.589 INFO:teuthology.orchestra.run.smithi031.stdout:55834574854 2024-09-13T09:07:09.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-09-13T09:07:09.596 DEBUG:teuthology.parallel:result is None 2024-09-13T09:07:09.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.5 2024-09-13T09:07:09.606 DEBUG:teuthology.parallel:result is None 2024-09-13T09:07:09.612 INFO:teuthology.orchestra.run.smithi031.stdout:64424509446 2024-09-13T09:07:09.625 INFO:teuthology.orchestra.run.smithi031.stdout:64424509445 2024-09-13T09:07:09.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-09-13T09:07:09.633 DEBUG:teuthology.parallel:result is None 2024-09-13T09:07:09.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.7 2024-09-13T09:07:09.643 DEBUG:teuthology.parallel:result is None 2024-09-13T09:07:09.643 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:07:09.643 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:07:09.972 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:09.973 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:07:09.989 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T09:07:09.047631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235384,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739336,"statfs":{"total":773094113280,"available":772853080064,"internally_reserved":0,"allocated":2867200,"data_stored":1743678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.935734"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151546+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:07:06.151546+0000","last_peered":"2024-09-13T09:07:06.151546+0000","last_clean":"2024-09-13T09:07:06.151546+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:07:06.151546+0000","last_undegraded":"2024-09-13T09:07:06.151546+0000","last_fullsized":"2024-09-13T09:07:06.151546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.247206+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:07:06.247206+0000","last_peered":"2024-09-13T09:07:06.247206+0000","last_clean":"2024-09-13T09:07:06.247206+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:07:06.247206+0000","last_undegraded":"2024-09-13T09:07:06.247206+0000","last_fullsized":"2024-09-13T09:07:06.247206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151545+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:07:06.151545+0000","last_peered":"2024-09-13T09:07:06.151545+0000","last_clean":"2024-09-13T09:07:06.151545+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:07:06.151545+0000","last_undegraded":"2024-09-13T09:07:06.151545+0000","last_fullsized":"2024-09-13T09:07:06.151545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.238908+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:07:06.238908+0000","last_peered":"2024-09-13T09:07:06.238908+0000","last_clean":"2024-09-13T09:07:06.238908+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:07:06.238908+0000","last_undegraded":"2024-09-13T09:07:06.238908+0000","last_fullsized":"2024-09-13T09:07:06.238908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151963+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:07:06.151963+0000","last_peered":"2024-09-13T09:07:06.151963+0000","last_clean":"2024-09-13T09:07:06.151963+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:07:06.151963+0000","last_undegraded":"2024-09-13T09:07:06.151963+0000","last_fullsized":"2024-09-13T09:07:06.151963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.906301+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:07:06.906301+0000","last_peered":"2024-09-13T09:07:06.906301+0000","last_clean":"2024-09-13T09:07:06.906301+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:07:06.906301+0000","last_undegraded":"2024-09-13T09:07:06.906301+0000","last_fullsized":"2024-09-13T09:07:06.906301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151050+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:07:06.151050+0000","last_peered":"2024-09-13T09:07:06.151050+0000","last_clean":"2024-09-13T09:07:06.151050+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:07:06.151050+0000","last_undegraded":"2024-09-13T09:07:06.151050+0000","last_fullsized":"2024-09-13T09:07:06.151050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.149435+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:07:06.149435+0000","last_peered":"2024-09-13T09:07:06.149435+0000","last_clean":"2024-09-13T09:07:06.149435+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:07:06.149435+0000","last_undegraded":"2024-09-13T09:07:06.149435+0000","last_fullsized":"2024-09-13T09:07:06.149435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T09:07:05.206385+0000","last_change":"2024-09-13T09:06:56.632919+0000","last_active":"2024-09-13T09:07:05.206385+0000","last_peered":"2024-09-13T09:07:05.206385+0000","last_clean":"2024-09-13T09:07:05.206385+0000","last_became_active":"2024-09-13T09:06:56.630117+0000","last_became_peered":"2024-09-13T09:06:56.630117+0000","last_unstale":"2024-09-13T09:07:05.206385+0000","last_undegraded":"2024-09-13T09:07:05.206385+0000","last_fullsized":"2024-09-13T09:07:05.206385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:32.893066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":212992,"data_stored":70382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":660731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:07:09.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:07:10.288 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:10.288 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:07:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T09:07:09.047631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235384,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739336,"statfs":{"total":773094113280,"available":772853080064,"internally_reserved":0,"allocated":2867200,"data_stored":1743678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.935734"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151546+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:07:06.151546+0000","last_peered":"2024-09-13T09:07:06.151546+0000","last_clean":"2024-09-13T09:07:06.151546+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:07:06.151546+0000","last_undegraded":"2024-09-13T09:07:06.151546+0000","last_fullsized":"2024-09-13T09:07:06.151546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.247206+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:07:06.247206+0000","last_peered":"2024-09-13T09:07:06.247206+0000","last_clean":"2024-09-13T09:07:06.247206+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:07:06.247206+0000","last_undegraded":"2024-09-13T09:07:06.247206+0000","last_fullsized":"2024-09-13T09:07:06.247206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151545+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:07:06.151545+0000","last_peered":"2024-09-13T09:07:06.151545+0000","last_clean":"2024-09-13T09:07:06.151545+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:07:06.151545+0000","last_undegraded":"2024-09-13T09:07:06.151545+0000","last_fullsized":"2024-09-13T09:07:06.151545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.238908+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:07:06.238908+0000","last_peered":"2024-09-13T09:07:06.238908+0000","last_clean":"2024-09-13T09:07:06.238908+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:07:06.238908+0000","last_undegraded":"2024-09-13T09:07:06.238908+0000","last_fullsized":"2024-09-13T09:07:06.238908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151963+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:07:06.151963+0000","last_peered":"2024-09-13T09:07:06.151963+0000","last_clean":"2024-09-13T09:07:06.151963+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:07:06.151963+0000","last_undegraded":"2024-09-13T09:07:06.151963+0000","last_fullsized":"2024-09-13T09:07:06.151963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.906301+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:07:06.906301+0000","last_peered":"2024-09-13T09:07:06.906301+0000","last_clean":"2024-09-13T09:07:06.906301+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:07:06.906301+0000","last_undegraded":"2024-09-13T09:07:06.906301+0000","last_fullsized":"2024-09-13T09:07:06.906301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151050+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:07:06.151050+0000","last_peered":"2024-09-13T09:07:06.151050+0000","last_clean":"2024-09-13T09:07:06.151050+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:07:06.151050+0000","last_undegraded":"2024-09-13T09:07:06.151050+0000","last_fullsized":"2024-09-13T09:07:06.151050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.149435+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:07:06.149435+0000","last_peered":"2024-09-13T09:07:06.149435+0000","last_clean":"2024-09-13T09:07:06.149435+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:07:06.149435+0000","last_undegraded":"2024-09-13T09:07:06.149435+0000","last_fullsized":"2024-09-13T09:07:06.149435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T09:07:05.206385+0000","last_change":"2024-09-13T09:06:56.632919+0000","last_active":"2024-09-13T09:07:05.206385+0000","last_peered":"2024-09-13T09:07:05.206385+0000","last_clean":"2024-09-13T09:07:05.206385+0000","last_became_active":"2024-09-13T09:06:56.630117+0000","last_became_peered":"2024-09-13T09:06:56.630117+0000","last_unstale":"2024-09-13T09:07:05.206385+0000","last_undegraded":"2024-09-13T09:07:05.206385+0000","last_fullsized":"2024-09-13T09:07:05.206385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:32.893066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":212992,"data_stored":70382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.337}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0660000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":660731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:07:10.306 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:07:10.306 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T09:07:10.307 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T09:07:10.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T09:07:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:10.626 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T09:07:10.644 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T09:07:10.644 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T09:07:10.655 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T09:07:10.655 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writeback'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}} 2024-09-13T09:07:10.655 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T09:07:10.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T09:07:10.972 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T09:07:10.972 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:07:10.973 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:07:11.255 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:11.256 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:07:11.272 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T09:07:11.048256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235384,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739336,"statfs":{"total":773094113280,"available":772853080064,"internally_reserved":0,"allocated":2867200,"data_stored":1743678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.927691"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151546+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:07:06.151546+0000","last_peered":"2024-09-13T09:07:06.151546+0000","last_clean":"2024-09-13T09:07:06.151546+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:07:06.151546+0000","last_undegraded":"2024-09-13T09:07:06.151546+0000","last_fullsized":"2024-09-13T09:07:06.151546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.247206+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:07:06.247206+0000","last_peered":"2024-09-13T09:07:06.247206+0000","last_clean":"2024-09-13T09:07:06.247206+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:07:06.247206+0000","last_undegraded":"2024-09-13T09:07:06.247206+0000","last_fullsized":"2024-09-13T09:07:06.247206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151545+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:07:06.151545+0000","last_peered":"2024-09-13T09:07:06.151545+0000","last_clean":"2024-09-13T09:07:06.151545+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:07:06.151545+0000","last_undegraded":"2024-09-13T09:07:06.151545+0000","last_fullsized":"2024-09-13T09:07:06.151545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.238908+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:07:06.238908+0000","last_peered":"2024-09-13T09:07:06.238908+0000","last_clean":"2024-09-13T09:07:06.238908+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:07:06.238908+0000","last_undegraded":"2024-09-13T09:07:06.238908+0000","last_fullsized":"2024-09-13T09:07:06.238908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151963+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:07:06.151963+0000","last_peered":"2024-09-13T09:07:06.151963+0000","last_clean":"2024-09-13T09:07:06.151963+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:07:06.151963+0000","last_undegraded":"2024-09-13T09:07:06.151963+0000","last_fullsized":"2024-09-13T09:07:06.151963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.906301+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:07:06.906301+0000","last_peered":"2024-09-13T09:07:06.906301+0000","last_clean":"2024-09-13T09:07:06.906301+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:07:06.906301+0000","last_undegraded":"2024-09-13T09:07:06.906301+0000","last_fullsized":"2024-09-13T09:07:06.906301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151050+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:07:06.151050+0000","last_peered":"2024-09-13T09:07:06.151050+0000","last_clean":"2024-09-13T09:07:06.151050+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:07:06.151050+0000","last_undegraded":"2024-09-13T09:07:06.151050+0000","last_fullsized":"2024-09-13T09:07:06.151050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.149435+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:07:06.149435+0000","last_peered":"2024-09-13T09:07:06.149435+0000","last_clean":"2024-09-13T09:07:06.149435+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:07:06.149435+0000","last_undegraded":"2024-09-13T09:07:06.149435+0000","last_fullsized":"2024-09-13T09:07:06.149435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T09:07:05.206385+0000","last_change":"2024-09-13T09:06:56.632919+0000","last_active":"2024-09-13T09:07:05.206385+0000","last_peered":"2024-09-13T09:07:05.206385+0000","last_clean":"2024-09-13T09:07:05.206385+0000","last_became_active":"2024-09-13T09:06:56.630117+0000","last_became_peered":"2024-09-13T09:06:56.630117+0000","last_unstale":"2024-09-13T09:07:05.206385+0000","last_undegraded":"2024-09-13T09:07:05.206385+0000","last_fullsized":"2024-09-13T09:07:05.206385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:32.893066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":212992,"data_stored":70382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":660731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:07:11.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:07:11.567 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:11.568 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:07:11.586 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T09:07:11.048256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235384,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739336,"statfs":{"total":773094113280,"available":772853080064,"internally_reserved":0,"allocated":2867200,"data_stored":1743678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":238014008},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.927691"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151546+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:07:06.151546+0000","last_peered":"2024-09-13T09:07:06.151546+0000","last_clean":"2024-09-13T09:07:06.151546+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:07:06.151546+0000","last_undegraded":"2024-09-13T09:07:06.151546+0000","last_fullsized":"2024-09-13T09:07:06.151546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.247206+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:07:06.247206+0000","last_peered":"2024-09-13T09:07:06.247206+0000","last_clean":"2024-09-13T09:07:06.247206+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:07:06.247206+0000","last_undegraded":"2024-09-13T09:07:06.247206+0000","last_fullsized":"2024-09-13T09:07:06.247206+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151545+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:07:06.151545+0000","last_peered":"2024-09-13T09:07:06.151545+0000","last_clean":"2024-09-13T09:07:06.151545+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:07:06.151545+0000","last_undegraded":"2024-09-13T09:07:06.151545+0000","last_fullsized":"2024-09-13T09:07:06.151545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.238908+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:07:06.238908+0000","last_peered":"2024-09-13T09:07:06.238908+0000","last_clean":"2024-09-13T09:07:06.238908+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:07:06.238908+0000","last_undegraded":"2024-09-13T09:07:06.238908+0000","last_fullsized":"2024-09-13T09:07:06.238908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151963+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:07:06.151963+0000","last_peered":"2024-09-13T09:07:06.151963+0000","last_clean":"2024-09-13T09:07:06.151963+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:07:06.151963+0000","last_undegraded":"2024-09-13T09:07:06.151963+0000","last_fullsized":"2024-09-13T09:07:06.151963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.906301+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:07:06.906301+0000","last_peered":"2024-09-13T09:07:06.906301+0000","last_clean":"2024-09-13T09:07:06.906301+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:07:06.906301+0000","last_undegraded":"2024-09-13T09:07:06.906301+0000","last_fullsized":"2024-09-13T09:07:06.906301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.151050+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:07:06.151050+0000","last_peered":"2024-09-13T09:07:06.151050+0000","last_clean":"2024-09-13T09:07:06.151050+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:07:06.151050+0000","last_undegraded":"2024-09-13T09:07:06.151050+0000","last_fullsized":"2024-09-13T09:07:06.151050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"19'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T09:07:06.149435+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:07:06.149435+0000","last_peered":"2024-09-13T09:07:06.149435+0000","last_clean":"2024-09-13T09:07:06.149435+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:07:06.149435+0000","last_undegraded":"2024-09-13T09:07:06.149435+0000","last_fullsized":"2024-09-13T09:07:06.149435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-13T09:07:05.206385+0000","last_change":"2024-09-13T09:06:56.632919+0000","last_active":"2024-09-13T09:07:05.206385+0000","last_peered":"2024-09-13T09:07:05.206385+0000","last_clean":"2024-09-13T09:07:05.206385+0000","last_became_active":"2024-09-13T09:06:56.630117+0000","last_became_peered":"2024-09-13T09:06:56.630117+0000","last_unstale":"2024-09-13T09:07:05.206385+0000","last_undegraded":"2024-09-13T09:07:05.206385+0000","last_fullsized":"2024-09-13T09:07:05.206385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:32.893066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":70382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]}]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":212992,"data_stored":70382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31328,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340512,"statfs":{"total":96636764160,"available":96604684288,"internally_reserved":0,"allocated":208896,"data_stored":70363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31900,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339940,"statfs":{"total":96636764160,"available":96604098560,"internally_reserved":0,"allocated":802816,"data_stored":660731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:07:11.587 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:07:11.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 21 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:fsid fe16bf6d-4166-4417-80fd-1123617a33d3 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-13T09:06:44.149006+0000 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-13T09:07:06.128047+0000 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 5 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-09-13T09:07:11.871 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client jewel 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release reef 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout: removed_snaps_queue [2~1] 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 8 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/3291756747,v1:172.21.15.31:6818/3291756747] [v2:172.21.15.31:6820/3291756747,v1:172.21.15.31:6822/3291756747] exists,up 33c6fc7d-9828-4500-b103-b96d1cd7f420 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 15 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/533520566,v1:172.21.15.31:6801/533520566] [v2:172.21.15.31:6802/533520566,v1:172.21.15.31:6803/533520566] exists,up 869ea3c5-8e6e-4b49-b70e-083f242a7aae 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6817/2253604482,v1:172.21.15.31:6819/2253604482] [v2:172.21.15.31:6821/2253604482,v1:172.21.15.31:6823/2253604482] exists,up e144d496-76ee-46e0-bdc6-135c77128535 2024-09-13T09:07:11.872 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/3367214693,v1:172.21.15.31:6809/3367214693] [v2:172.21.15.31:6810/3367214693,v1:172.21.15.31:6811/3367214693] exists,up 696b9414-9c19-4984-a8ff-14548e46b357 2024-09-13T09:07:11.873 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6800/3778074750,v1:172.21.15.60:6801/3778074750] [v2:172.21.15.60:6802/3778074750,v1:172.21.15.60:6803/3778074750] exists,up ac58040d-9b54-42ca-b143-1816abb54816 2024-09-13T09:07:11.873 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6808/1926603087,v1:172.21.15.60:6809/1926603087] [v2:172.21.15.60:6811/1926603087,v1:172.21.15.60:6813/1926603087] exists,up e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2 2024-09-13T09:07:11.873 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6820/766305500,v1:172.21.15.60:6821/766305500] [v2:172.21.15.60:6823/766305500,v1:172.21.15.60:6825/766305500] exists,up d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d 2024-09-13T09:07:11.873 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/1032956039,v1:172.21.15.60:6812/1032956039] [v2:172.21.15.60:6814/1032956039,v1:172.21.15.60:6816/1032956039] exists,up f36c332c-9497-47e8-b943-f6b6dcaebeb8 2024-09-13T09:07:11.888 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/3291756747,v1:172.21.15.31:6818/3291756747] [v2:172.21.15.31:6820/3291756747,v1:172.21.15.31:6822/3291756747] exists,up 33c6fc7d-9828-4500-b103-b96d1cd7f420', 'osd.1 up in weight 1 up_from 15 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/533520566,v1:172.21.15.31:6801/533520566] [v2:172.21.15.31:6802/533520566,v1:172.21.15.31:6803/533520566] exists,up 869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6817/2253604482,v1:172.21.15.31:6819/2253604482] [v2:172.21.15.31:6821/2253604482,v1:172.21.15.31:6823/2253604482] exists,up e144d496-76ee-46e0-bdc6-135c77128535', 'osd.3 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/3367214693,v1:172.21.15.31:6809/3367214693] [v2:172.21.15.31:6810/3367214693,v1:172.21.15.31:6811/3367214693] exists,up 696b9414-9c19-4984-a8ff-14548e46b357', 'osd.4 up in weight 1 up_from 14 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6800/3778074750,v1:172.21.15.60:6801/3778074750] [v2:172.21.15.60:6802/3778074750,v1:172.21.15.60:6803/3778074750] exists,up ac58040d-9b54-42ca-b143-1816abb54816', 'osd.5 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6808/1926603087,v1:172.21.15.60:6809/1926603087] [v2:172.21.15.60:6811/1926603087,v1:172.21.15.60:6813/1926603087] exists,up e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6820/766305500,v1:172.21.15.60:6821/766305500] [v2:172.21.15.60:6823/766305500,v1:172.21.15.60:6825/766305500] exists,up d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'osd.7 up in weight 1 up_from 15 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/1032956039,v1:172.21.15.60:6812/1032956039] [v2:172.21.15.60:6814/1032956039,v1:172.21.15.60:6816/1032956039] exists,up f36c332c-9497-47e8-b943-f6b6dcaebeb8'] 2024-09-13T09:07:11.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T09:07:12.002 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:12.002 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mon.a", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster": "ceph", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket_mode": "", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_client_required": "cephx, none", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_cluster_required": "cephx", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_debug": "true", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_required": "cephx", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_supported": "cephx", 2024-09-13T09:07:12.003 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio": "true", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_reap_max": "16", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_async_discard": "false", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_block_size": "4096", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio": "false", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_enable_discard": "false", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry": "3", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T09:07:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash": "0", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring": "false", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_hipri": "false", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_type": "", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_allocator": "hybrid", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_buffered_io": "true", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T09:07:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery": "false", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T09:07:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_sync_write": "false", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocator": "hybrid", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T09:07:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_create": "true", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_create": "false", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_path": "", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_size": "0", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_path": "", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_size": "107374182400", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_create": "false", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_path": "", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs": "true", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T09:07:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune": "true", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size": "0", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T09:07:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_type": "2q", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_clone_cow": "true", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_mode": "none", 2024-09-13T09:07:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_freelist": "false", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T09:07:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T09:07:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fail_eio": "false", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T09:07:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T09:07:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T09:07:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T09:07:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T09:07:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T09:07:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T09:07:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T09:07:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T09:07:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size": "0", 2024-09-13T09:07:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T09:07:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_mem": "512", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T09:07:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_tracing": "false", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T09:07:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T09:07:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_version": "2", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_signatures": "false", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_version": "2", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_signatures": "false", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_version": "2", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_sign_messages": "true", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "chdir": "", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_acl_type": "", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_asio_thread_count": "2", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_mid": "0.750000", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_size": "16384", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_caps_release_delay": "5", 2024-09-13T09:07:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_check_pool_perm": "true", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_force_sync_read": "false", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_getattr_caps": "false", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_features": "", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_remount": "false", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_dirsize_rbytes": "true", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_force_lazyio": "false", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_fs": "", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_release_failure": "false", 2024-09-13T09:07:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_inline_size": "4096", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mds_namespace": "", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_metadata": "", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_gid": "-1", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_timeout": "300", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_uid": "-1", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mountpoint": "/", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_notify_timeout": "10", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc": "true", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_objects": "1000", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_size": "209715200", 2024-09-13T09:07:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_permissions": "true", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota": "true", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota_df": "true", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_bytes": "0", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_periods": "4", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_min": "131072", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_reconnect_stale": "false", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_shutdown_timeout": "30", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_snapdir": ".snap", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_tick_interval": "1", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_trace": "", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T09:07:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_faked_inos": "false", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_random_mds": "false", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog": "false", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_port": "12201", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_monitors": "default=true", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog": "false", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_level": "info", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "-", 2024-09-13T09:07:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network": "", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network_interface": "", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_isal": "false", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_level": "5", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zstd_level": "1", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-13T09:07:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_smp": "1", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location": "", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook": "", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook_timeout": "10", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "daemonize": "false", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "daos_pool": "tank", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_alienstore": "0/5", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok": "1/5", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok_assert_abort": "false", 2024-09-13T09:07:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_auth": "20/20", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bdev": "1/3", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluefs": "1/5", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluestore": "1/5", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_buffer": "0/1", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephsqlite": "0/5", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_client": "0/5", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_compressor": "1/5", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_context": "0/1", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crush": "1/1", 2024-09-13T09:07:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crypto": "1/5", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cyanstore": "0/5", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_dpdk": "1/5", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_eventtrace": "1/5", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filer": "0/1", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filestore": "1/3", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_finisher": "1/1", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_fuse": "1/5", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_javaclient": "1/5", 2024-09-13T09:07:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journal": "1/3", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journaler": "0/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_kstore": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_leveldb": "4/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_lockdep": "0/1", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mclock": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_balancer": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_locker": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_migrator": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_memstore": "1/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgr": "2/5", 2024-09-13T09:07:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgrc": "1/5", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mon": "20/20", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_monc": "0/10", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ms": "1/1", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_none": "0/5", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objclass": "0/5", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objectcacher": "0/5", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objecter": "0/1", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_optracker": "0/5", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_osd": "1/5", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_paxos": "20/20", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_perfcounter": "1/5", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_prioritycache": "1/5", 2024-09-13T09:07:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rados": "0/5", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd": "0/5", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_replay": "0/5", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_refs": "0/0", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_reserver": "1/1", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw": "1/5", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_access": "1/5", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T09:07:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T09:07:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_flight": "1/5", 2024-09-13T09:07:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_sync": "1/5", 2024-09-13T09:07:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rocksdb": "4/5", 2024-09-13T09:07:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore": "0/5", 2024-09-13T09:07:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_backref": "0/5", 2024-09-13T09:07:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cache": "0/5", 2024-09-13T09:07:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T09:07:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_device": "0/5", 2024-09-13T09:07:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_epm": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_journal": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_lba": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_odata": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_omap": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_onode": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_t": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_tm": "0/5", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_striper": "0/1", 2024-09-13T09:07:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_test": "0/5", 2024-09-13T09:07:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_throttle": "1/1", 2024-09-13T09:07:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_timer": "0/1", 2024-09-13T09:07:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_tp": "0/5", 2024-09-13T09:07:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T09:07:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "device_failure_prediction_mode": "none", 2024-09-13T09:07:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T09:07:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-13T09:07:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_graylog": "false", 2024-09-13T09:07:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_journald": "false", 2024-09-13T09:07:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_stderr": "true", 2024-09-13T09:07:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_syslog": "false", 2024-09-13T09:07:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event_tracing": "false", 2024-09-13T09:07:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T09:07:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_http_port": "9926", 2024-09-13T09:07:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_prio_limit": "5", 2024-09-13T09:07:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T09:07:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sort_metrics": "true", 2024-09-13T09:07:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_stats_period": "5", 2024-09-13T09:07:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "fake_statfs_for_testing": "0", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "fatal_signal_handlers": "true", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_purge_ops": "10", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_truncate_ops": "128", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_blackhole": "false", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_snap": "true", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_caller_concurrency": "10", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T09:07:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T09:07:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T09:07:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_omap_check": "false", 2024-09-13T09:07:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-13T09:07:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_verify_split": "false", 2024-09-13T09:07:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_dump_file": "", 2024-09-13T09:07:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T09:07:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T09:07:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fadvise": "true", 2024-09-13T09:07:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fail_eio": "true", 2024-09-13T09:07:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T09:07:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_size": "128", 2024-09-13T09:07:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap": "false", 2024-09-13T09:07:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T09:07:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T09:07:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_inject_stall": "0", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_parallel": "false", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_trailing": "false", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_writeahead": "false", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_kill_at": "0", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T09:07:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T09:07:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T09:07:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T09:07:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T09:07:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T09:07:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T09:07:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T09:07:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T09:07:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T09:07:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T09:07:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_merge_threshold": "-10", 2024-09-13T09:07:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_odsync_write": "false", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend_path": "", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_threads": "2", 2024-09-13T09:07:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_punch_hole": "false", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_ops": "50", 2024-09-13T09:07:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_seek_data_hole": "false", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc": "false", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_splice": "false", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_multiple": "2", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_rand_factor": "20", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_update_to": "1000", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T09:07:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T09:07:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T09:07:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T09:07:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T09:07:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T09:07:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T09:07:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T09:07:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T09:07:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T09:07:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T09:07:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T09:07:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_xfs_extsize": "false", 2024-09-13T09:07:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_zfs_snap": "false", 2024-09-13T09:07:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "fio_dir": "/tmp/fio", 2024-09-13T09:07:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:07:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_allow_other": "true", 2024-09-13T09:07:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T09:07:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_big_writes": "true", 2024-09-13T09:07:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_debug": "false", 2024-09-13T09:07:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_default_permissions": "false", 2024-09-13T09:07:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_disable_pagecache": "false", 2024-09-13T09:07:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_max_write": "0", 2024-09-13T09:07:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_multithreaded": "true", 2024-09-13T09:07:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_require_active_mds": "true", 2024-09-13T09:07:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_set_user_groups": "true", 2024-09-13T09:07:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_move": "true", 2024-09-13T09:07:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_read": "true", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_write": "true", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_target_name": "ceph", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_file": "", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_inject_failure": "0", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_interval": "5", 2024-09-13T09:07:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "host": "smithi031", 2024-09-13T09:07:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T09:07:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T09:07:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T09:07:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T09:07:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T09:07:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "inject_early_sigterm": "false", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_agent_port": "6799", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_tracing_enable": "false", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_aio": "true", 2024-09-13T09:07:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_align_min_size": "65536", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_align": "true", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_size": "4096", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_dio": "true", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_discard": "false", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_force_aio": "false", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_ignore_corruption": "false", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_entries": "100", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_replay_from": "0", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T09:07:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_write_header_frequency": "0", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_zero_on_create": "false", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prefetch_periods": "10", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prezero_periods": "5", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_write_head_interval": "15", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "keyfile": "", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_backend": "rocksdb", 2024-09-13T09:07:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_bytes": "67108864", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_ops": "512", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_onode_map_size": "1024", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T09:07:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_transaction": "false", 2024-09-13T09:07:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_block_size": "65536", 2024-09-13T09:07:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_bloom_size": "0", 2024-09-13T09:07:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_cache_size": "536870912", 2024-09-13T09:07:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_compact_on_mount": "false", 2024-09-13T09:07:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_compression": "false", 2024-09-13T09:07:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_log": "", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_max_open_files": "0", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_paranoid": "false", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "librados_thread_count": "2", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep": "false", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep_force_backtrace": "false", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "log_coarse_timestamps": "true", 2024-09-13T09:07:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_flush_on_exit": "false", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_port": "12201", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_new": "1000", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_recent": "10000", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stderr_prefix": "", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": "true", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_graylog": "false", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_journald": "false", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_stderr": "false", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_syslog": "false", 2024-09-13T09:07:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_action_on_write_error": "1", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_alternate_name_max": "8192", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_asio_thread_count": "2", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_export_pin": "true", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T09:07:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T09:07:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_interval": "10", 2024-09-13T09:07:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max": "-1", 2024-09-13T09:07:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max_until": "-1", 2024-09-13T09:07:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_merge_size": "50", 2024-09-13T09:07:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T09:07:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T09:07:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T09:07:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T09:07:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_mode": "0", 2024-09-13T09:07:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T09:07:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T09:07:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T09:07:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T09:07:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_bits": "3", 2024-09-13T09:07:12.063 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T09:07:12.063 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_size": "10000", 2024-09-13T09:07:12.063 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T09:07:12.063 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T09:07:12.063 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_mid": "0.700000", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T09:07:12.064 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_interval": "1", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_auth_pins": "false", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_frag": "false", 2024-09-13T09:07:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_scatterstat": "false", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_subtrees": "false", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_default_dir_hash": "2", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_defer_session_stale": "true", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_entries": "0", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_prefetch": "true", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T09:07:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T09:07:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T09:07:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T09:07:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_early_reply": "true", 2024-09-13T09:07:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enable_op_tracker": "true", 2024-09-13T09:07:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enforce_unique_name": "true", 2024-09-13T09:07:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T09:07:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T09:07:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T09:07:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T09:07:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T09:07:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T09:07:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_health_dummy": "false", 2024-09-13T09:07:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_join_fs": "", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_journal_format": "1", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_create_at": "0", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_export_at": "0", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_import_at": "0", 2024-09-13T09:07:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_at": "0", 2024-09-13T09:07:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T09:07:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T09:07:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_link_at": "0", 2024-09-13T09:07:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T09:07:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_openc_at": "0", 2024-09-13T09:07:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_rename_at": "0", 2024-09-13T09:07:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T09:07:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_events": "-1", 2024-09-13T09:07:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_segments": "128", 2024-09-13T09:07:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_pause": "false", 2024-09-13T09:07:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_segment_size": "0", 2024-09-13T09:07:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T09:07:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T09:07:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T09:07:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T09:07:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_requests": "100000", 2024-09-13T09:07:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_export_size": "20971520", 2024-09-13T09:07:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_file_recover": "32", 2024-09-13T09:07:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_files": "64", 2024-09-13T09:07:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops": "8192", 2024-09-13T09:07:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T09:07:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T09:07:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T09:07:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_metrics_update_interval": "2", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_per_client": "100", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_numa_node": "-1", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_duration": "600", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_size": "20", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T09:07:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_log_threshold": "5", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_grace": "15", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_interval": "5", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_caps": "30000", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T09:07:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_interval": "1.000000", 2024-09-13T09:07:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T09:07:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T09:07:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_gid": "0", 2024-09-13T09:07:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_uid": "0", 2024-09-13T09:07:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T09:07:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T09:07:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T09:07:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T09:07:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T09:07:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T09:07:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T09:07:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T09:07:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T09:07:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T09:07:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_shutdown_check": "0", 2024-09-13T09:07:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_skip_ino": "0", 2024-09-13T09:07:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T09:07:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T09:07:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_min_uid": "0", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_rstat": "false", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_symlink_recovery": "true", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_exports": "0", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_fragments": "0", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_tick_interval": "5.000000", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_valgrind_exit": "false", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_backtrace": "1", 2024-09-13T09:07:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_scatter": "false", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_sessions": "false", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool_debug": "false", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_set": "false", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_size": "65536", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_bytes": "134217728", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_messages": "512", 2024-09-13T09:07:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T09:07:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T09:07:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T09:07:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T09:07:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_disabled_modules": "", 2024-09-13T09:07:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T09:07:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T09:07:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T09:07:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_messages": "128", 2024-09-13T09:07:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T09:07:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T09:07:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_messages": "128", 2024-09-13T09:07:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_messages": "8192", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_pool": "true", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_standby_modules": "true", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_period": "5", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_threshold": "5", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_tick_period": "2", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T09:07:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_delete": "true", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_bytes": "104857600", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T09:07:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file_level": "debug", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T09:07:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T09:07:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T09:07:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T09:07:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-13T09:07:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T09:07:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_start": "false", 2024-09-13T09:07:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_trim": "true", 2024-09-13T09:07:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T09:07:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T09:07:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T09:07:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cpu_threads": "4", 2024-09-13T09:07:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T09:07:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T09:07:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T09:07:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_crit": "5", 2024-09-13T09:07:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_warn": "5", 2024-09-13T09:07:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T09:07:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T09:07:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_json": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_extra_checks": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_quincy": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T09:07:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_default_strategy": "1", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_timeout": "5.000000", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_enable_op_tracker": "true", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fake_pool_delete": "false", 2024-09-13T09:07:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_force_quorum_join": "false", 2024-09-13T09:07:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T09:07:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T09:07:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_log_update_period": "5", 2024-09-13T09:07:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_max_detail": "50", 2024-09-13T09:07:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog": "true", 2024-09-13T09:07:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T09:07:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T09:07:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host": "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.60", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host_override": "", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_initial_members": "", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease": "5.000000", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T09:07:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_full_interval": "50", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max": "10000", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max_summary": "50", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_epochs": "500", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_osd": "10000", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T09:07:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_autotune": "true", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_target": "2147483648", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_digest_period": "5", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_min_osdmap_epochs": "500", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_complaint_time": "30", 2024-09-13T09:07:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_duration": "600", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_size": "20", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_log_threshold": "5", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size": "500", 2024-09-13T09:07:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T09:07:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T09:07:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T09:07:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-13T09:07:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T09:07:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T09:07:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T09:07:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T09:07:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_report_timeout": "900", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-13T09:07:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_osds": "4", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T09:07:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_interval": "86400", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_max_keys": "100", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_timeout": "300", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_session_timeout": "300", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_smart_report_timeout": "5", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_size": "4", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T09:07:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_debug": "false", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_tick_interval": "5", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T09:07:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T09:07:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_older_version": "true", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "monmap": "", 2024-09-13T09:07:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T09:07:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T09:07:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T09:07:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T09:07:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T09:07:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T09:07:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_tracing_enabled": "false", 2024-09-13T09:07:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_op_threads": "3", 2024-09-13T09:07:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T09:07:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_cm": "false", 2024-09-13T09:07:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_device_name": "", 2024-09-13T09:07:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T09:07:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T09:07:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T09:07:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T09:07:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T09:07:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T09:07:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T09:07:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T09:07:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T09:07:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T09:07:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_sl": "3", 2024-09-13T09:07:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T09:07:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_type": "ib", 2024-09-13T09:07:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_reap_threshold": "5", 2024-09-13T09:07:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_before_connect": "false", 2024-09-13T09:07:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv4": "true", 2024-09-13T09:07:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv6": "false", 2024-09-13T09:07:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr1": "true", 2024-09-13T09:07:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr2": "true", 2024-09-13T09:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_max": "7568", 2024-09-13T09:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_min": "6800", 2024-09-13T09:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T09:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_count": "3", 2024-09-13T09:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_delay": "5", 2024-09-13T09:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_client": "false", 2024-09-13T09:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mds": "false", 2024-09-13T09:07:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mgr": "false", 2024-09-13T09:07:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mon": "false", 2024-09-13T09:07:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_osd": "false", 2024-09-13T09:07:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_client_mode": "crc secure", 2024-09-13T09:07:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T09:07:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_type": "", 2024-09-13T09:07:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_compress_secure": "false", 2024-09-13T09:07:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T09:07:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T09:07:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_data": "true", 2024-09-13T09:07:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_header": "true", 2024-09-13T09:07:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T09:07:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bug": "true", 2024-09-13T09:07:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_old_message": "true", 2024-09-13T09:07:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T09:07:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T09:07:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T09:07:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T09:07:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T09:07:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T09:07:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T09:07:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T09:07:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T09:07:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hugepages": "", 2024-09-13T09:07:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T09:07:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T09:07:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_lro": "true", 2024-09-13T09:07:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T09:07:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T09:07:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_pmd": "", 2024-09-13T09:07:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_port_id": "0", 2024-09-13T09:07:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T09:07:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T09:07:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_on_send": "false", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_type": "", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_network_congestion": "0", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T09:07:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_accept_failures": "4", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_backoff": "15.000000", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_mode": "none", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_min_cost": "65536", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_public_type": "", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_service_mode": "crc secure", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T09:07:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_nodelay": "true", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_type": "async+posix", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "no_config_file": "false", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_ops": "1024", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_timeout": "10.000000", 2024-09-13T09:07:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_blackhole": "false", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "openssl_engine_opts": "", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_ops": "4", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_slop": "0.020000", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_asio_thread_count": "2", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T09:07:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_max": "512", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_min": "64", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_peering": "false", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_beacon_report_interval": "300", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_duration": "30", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T09:07:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blkin_trace_all": "false", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_update_on_start": "true", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_cap": "256", 2024-09-13T09:07:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_op_priority": "63", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_watch_timeout": "30", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_command_max_records": "256", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_compact_on_start": "false", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_on_start": "true", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T09:07:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T09:07:12.123 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T09:07:12.123 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T09:07:12.123 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T09:07:12.123 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T09:07:12.124 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T09:07:12.124 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T09:07:12.124 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T09:07:12.124 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T09:07:12.124 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T09:07:12.124 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_op_order": "false", 2024-09-13T09:07:12.124 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_shutdown": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T09:07:12.125 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_notify_timeout": "30", 2024-09-13T09:07:12.126 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_enable_op_tracker": "true", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_info": "true", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown": "true", 2024-09-13T09:07:12.127 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T09:07:12.128 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T09:07:12.128 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T09:07:12.128 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T09:07:12.128 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T09:07:12.128 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_function_tracing": "false", 2024-09-13T09:07:12.128 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_grace": "20", 2024-09-13T09:07:12.128 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_interval": "6", 2024-09-13T09:07:12.128 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_stale": "600", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T09:07:12.129 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_size": "5120", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_kill_backfill_at": "0", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_cache_size": "50", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_dedup": "true", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max": "40", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_name_len": "100", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_size": "0", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_backfills": "1", 2024-09-13T09:07:12.130 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_count": "5", 2024-09-13T09:07:12.131 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_period": "600", 2024-09-13T09:07:12.131 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_name_len": "2048", 2024-09-13T09:07:12.131 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T09:07:12.131 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_size": "134217728", 2024-09-13T09:07:12.131 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T09:07:12.131 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T09:07:12.131 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T09:07:12.131 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T09:07:12.132 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T09:07:12.132 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pgls": "1024", 2024-09-13T09:07:12.132 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_cost": "8388608", 2024-09-13T09:07:12.132 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_objects": "10", 2024-09-13T09:07:12.132 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_scrubs": "3", 2024-09-13T09:07:12.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T09:07:12.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T09:07:12.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T09:07:12.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_size": "90", 2024-09-13T09:07:12.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T09:07:12.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T09:07:12.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T09:07:12.133 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_profile": "balanced", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T09:07:12.134 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_base": "805306368", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T09:07:12.135 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target": "4294967296", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_autotune": "false", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_recovery_priority": "0", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_interval": "5", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_cache_shards": "32", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T09:07:12.136 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T09:07:12.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_node": "-1", 2024-09-13T09:07:12.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T09:07:12.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T09:07:12.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objecter_finishers": "1", 2024-09-13T09:07:12.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore": "bluestore", 2024-09-13T09:07:12.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_fuse": "false", 2024-09-13T09:07:12.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_tracing": "false", 2024-09-13T09:07:12.137 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_duration": "600", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_size": "20", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_log_threshold": "5", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards": "0", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T09:07:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_timeout": "15", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_open_classes_on_start": "true", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_os_flags": "0", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_peering_op_priority": "255", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_priority": "5", 2024-09-13T09:07:12.139 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T09:07:12.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T09:07:12.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T09:07:12.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T09:07:12.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T09:07:12.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T09:07:12.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T09:07:12.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-13T09:07:12.140 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crimson": "false", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T09:07:12.141 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flags": "0", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_min_size": "0", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_size": "2", 2024-09-13T09:07:12.142 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_type": "replicated", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_cost": "20971520", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active": "0", 2024-09-13T09:07:12.143 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_priority": "3", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_priority": "5", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T09:07:12.144 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_repair_during_recovery": "false", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T09:07:12.145 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_cost": "52428800", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_hour": "0", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T09:07:12.146 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T09:07:12.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T09:07:12.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_priority": "5", 2024-09-13T09:07:12.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-13T09:07:12.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T09:07:12.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-13T09:07:12.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T09:07:12.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T09:07:12.147 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T09:07:12.148 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_skip_data_digest": "false", 2024-09-13T09:07:12.148 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_smart_report_timeout": "5", 2024-09-13T09:07:12.148 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T09:07:12.148 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_priority": "5", 2024-09-13T09:07:12.148 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T09:07:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T09:07:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T09:07:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T09:07:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T09:07:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T09:07:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T09:07:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_transaction_size": "30", 2024-09-13T09:07:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tracing": "false", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_use_stale_snap": "false", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T09:07:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_kill_at": "0", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_max_join_drift": "10", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min": "500", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min_wait": "0.050000", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max": "500", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_min": "250", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_stash_full_interval": "25", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_max": "500", 2024-09-13T09:07:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_min": "250", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "perf": "true", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "-", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrv": "", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "public_bind_addr": "-", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network": "", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network_interface": "", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_enabled": "false", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T09:07:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_mon_op_timeout": "0", 2024-09-13T09:07:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_osd_op_timeout": "0", 2024-09-13T09:07:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_tracing": "false", 2024-09-13T09:07:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_atime_update_interval": "60", 2024-09-13T09:07:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T09:07:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T09:07:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T09:07:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T09:07:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T09:07:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T09:07:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache": "true", 2024-09-13T09:07:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T09:07:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T09:07:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_policy": "writearound", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_size": "33554432", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_compression_hint": "none", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T09:07:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_clone_format": "auto", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_data_pool": "", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_features": "61", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_format": "2", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_map_options": "", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_order": "22", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_pool": "rbd", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_count": "0", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T09:07:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler": "simple", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T09:07:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T09:07:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T09:07:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T09:07:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T09:07:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_order": "24", 2024-09-13T09:07:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_pool": "", 2024-09-13T09:07:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_splay_width": "4", 2024-09-13T09:07:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T09:07:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T09:07:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T09:07:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T09:07:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T09:07:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T09:07:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T09:07:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_threads": "1", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T09:07:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_plugins": "", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T09:07:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T09:07:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_tracing": "false", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_names": "true", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_pool": "true", 2024-09-13T09:07:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_base_url": "", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_log_level": "", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_admin_entry": "admin", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_backend_store": "rados", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_barbican_url": "", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_beast_enable_async": "true", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T09:07:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_enabled": "true", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_lru_size": "10000", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_config_store": "rados", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_content_length_compat": "false", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cross_domain_policy": "", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T09:07:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T09:07:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T09:07:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T09:07:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_window": "30", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T09:07:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T09:07:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_name": "", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_s3website_name": "", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T09:07:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_ops_log": "false", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_static_website": "false", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_usage_log": "false", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_expose_bucket": "false", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_extended_http_attrs": "", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_filter": "none", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T09:07:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_objs": "32", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_init_timeout": "300", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T09:07:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_project": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_user": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_api_version": "2", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T09:07:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_url": "", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_objs": "32", 2024-09-13T09:07:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_worker": "3", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_http_headers": "", 2024-09-13T09:07:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_size": "0", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_listing_results": "1000", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_notify_retries": "10", 2024-09-13T09:07:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T09:07:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T09:07:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_control_oids": "8", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_numa_node": "-1", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_tracing": "false", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_token": "", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_url": "", 2024-09-13T09:07:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_rados": "false", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_continue": "true", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T09:07:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_tracing": "false", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm": "", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_id": "", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region": "", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T09:07:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_request_uri": "", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_resolve_cname": "false", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_run_sync_thread": "true", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T09:07:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_script_uri": "", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_service_provider_name": "", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_id": "", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_secret": "", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_entry": "sts", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_key": "sts", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T09:07:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_url": "", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_custom_header": "", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_need_stats": "true", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_tenant_name": "", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url": "", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T09:07:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_lease_period": "120", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_thread_pool_size": "512", 2024-09-13T09:07:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_comment": "", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_createby": "", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_encoding": "", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_flag": "false", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_origin": "", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_tracker": "", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_shards": "32", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_use_opa_authz": "false", 2024-09-13T09:07:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_unique_email": "true", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_verify_ssl": "true", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone": "", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_id": "", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T09:07:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup": "", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_id": "", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_block_size": "4096", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T09:07:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_partition_filters": "false", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_perf": "false", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T09:07:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "run_dir": "/var/run/ceph", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_block_create": "true", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_device_size": "53687091200", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_main_device_type": "SSD", 2024-09-13T09:07:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_segment_size": "67108864", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "set_keepcaps": "false", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "setgroup": "", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser": "", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser_match_path": "", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "thp": "false", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_default_timeout": "60", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T09:07:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "throttler_perf_counter": "true" 2024-09-13T09:07:12.195 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:12.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T09:07:12.346 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-13T09:07:12.346 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:07:12.359 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-13T09:07:12.359 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:07:12.368 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-13T09:07:12.368 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:07:12.384 DEBUG:teuthology.orchestra.run.smithi060:> type ceph-objectstore-tool 2024-09-13T09:07:12.389 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:07:12.389 DEBUG:teuthology.orchestra.run.smithi031:> type ceph-objectstore-tool 2024-09-13T09:07:12.393 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:07:12.394 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-13T09:07:12.407 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-13T09:07:12.407 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T09:07:12.407 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:12.407 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:07:12.408 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-13T09:07:12.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-13T09:07:12.408 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T09:07:12.408 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T09:07:12.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:12.409 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T09:07:12.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:12.409 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T09:07:12.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:07:12.410 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-13T09:07:12.433 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:12.430+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:12.457 INFO:teuthology.orchestra.run.smithi031.stdout:mapped writes DISABLED 2024-09-13T09:07:12.457 INFO:teuthology.orchestra.run.smithi031.stdout:mapped reads DISABLED 2024-09-13T09:07:12.458 INFO:teuthology.orchestra.run.smithi031.stdout:Seed set to 6805 2024-09-13T09:07:12.510 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:12.505+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:12.553 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:12.553 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:12.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:12.593 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:12.593 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:12.610 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:12.610 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:12.610 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:12.609+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:12.626 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:12.626 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:12.643 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:12.643 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:12.663 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:12.663 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:12.683 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:12.683 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:12.703 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:12.703 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:12.704 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:12.704 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:12.712 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:12.706+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:12.718 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:12.723 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:12.724 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:12.741 INFO:teuthology.orchestra.run.smithi031.stdout:moved 0 / 18 (0%) 2024-09-13T09:07:12.742 INFO:teuthology.orchestra.run.smithi031.stdout:avg 2.25 2024-09-13T09:07:12.742 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 1.39194 -> 1.39194 (expected baseline 1.40312) 2024-09-13T09:07:12.742 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.1 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2024-09-13T09:07:12.742 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.3 with 5 -> 5 pgs (2.22222 -> 2.22222 * mean) 2024-09-13T09:07:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:12.743 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-13T09:07:12.743 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 0.05 2024-09-13T09:07:12.743 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-13T09:07:12.743 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0003 2024-09-13T09:07:12.743 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0003 2024-09-13T09:07:12.743 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-13T09:07:12.758 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-13T09:07:12.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-13T09:07:12.812 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:12.810+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:12.832 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:12.832 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:12.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:12.939 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:12.937+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:13.015 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:13.010+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:13.018 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:13.018 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:13.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:13.094 INFO:teuthology.orchestra.run.smithi031.stdout:moved 0 / 18 (0%) 2024-09-13T09:07:13.094 INFO:teuthology.orchestra.run.smithi031.stdout:avg 2.25 2024-09-13T09:07:13.094 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 1.39194 -> 1.39194 (expected baseline 1.40312) 2024-09-13T09:07:13.094 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.1 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2024-09-13T09:07:13.094 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.3 with 5 -> 5 pgs (2.22222 -> 2.22222 * mean) 2024-09-13T09:07:13.095 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:13.095 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-13T09:07:13.095 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 0.05 2024-09-13T09:07:13.095 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-13T09:07:13.095 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0003 2024-09-13T09:07:13.095 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0003 2024-09-13T09:07:13.095 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-13T09:07:13.111 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-13T09:07:13.112 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-13T09:07:13.115 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:13.110+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:13.166 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:07:13.204 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:13.205 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:13.216 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:13.213+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:13.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:13.317 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:13.314+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:13.372 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:13.372 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:13.385 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:13.431 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:13.429+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:13.437 INFO:teuthology.orchestra.run.smithi031.stdout:moved 0 / 82 (0%) 2024-09-13T09:07:13.437 INFO:teuthology.orchestra.run.smithi031.stdout:avg 10.25 2024-09-13T09:07:13.437 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-13T09:07:13.437 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-13T09:07:13.438 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-13T09:07:13.438 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:13.438 INFO:teuthology.orchestra.run.smithi031.stdout:oload 1000 2024-09-13T09:07:13.438 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 0.05 2024-09-13T09:07:13.438 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-13T09:07:13.438 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0003 2024-09-13T09:07:13.438 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0031 2024-09-13T09:07:13.438 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-13T09:07:13.455 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-13T09:07:13.455 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-13T09:07:13.495 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:13.495 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:13.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:13.519 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:13.514+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:13.620 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:13.617+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:13.665 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:13.666 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:13.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:13.721 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:13.718+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:13.790 INFO:teuthology.orchestra.run.smithi031.stdout:moved 0 / 82 (0%) 2024-09-13T09:07:13.790 INFO:teuthology.orchestra.run.smithi031.stdout:avg 10.25 2024-09-13T09:07:13.790 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-13T09:07:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-13T09:07:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-13T09:07:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:13.791 INFO:teuthology.orchestra.run.smithi031.stdout:oload 1000 2024-09-13T09:07:13.792 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 3 2024-09-13T09:07:13.792 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-13T09:07:13.792 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0003 2024-09-13T09:07:13.792 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0031 2024-09-13T09:07:13.792 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-13T09:07:13.808 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-13T09:07:13.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 1.0 2024-09-13T09:07:13.833 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:13.833 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:13.834 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:13.829+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:13.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:13.939 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:13.937+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:14.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:14.063 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:14.061+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:14.100 INFO:teuthology.orchestra.run.smithi031.stdout:moved 0 / 82 (0%) 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:avg 10.25 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:stddev 2.4367 -> 2.4367 (expected baseline 2.99479) 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:min osd.7 with 6 -> 6 pgs (0.585366 -> 0.585366 * mean) 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:max osd.3 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:oload 110 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:max_change 1 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:max_change_osds 4 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:average_utilization 0.0003 2024-09-13T09:07:14.101 INFO:teuthology.orchestra.run.smithi031.stdout:overload_utilization 0.0003 2024-09-13T09:07:14.102 INFO:teuthology.orchestra.run.smithi031.stderr:no change 2024-09-13T09:07:14.125 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:14.122+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:14.178 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:14.178 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:14.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:14.226 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:14.222+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:14.312 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:14.312 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:14.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:14.327 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:14.325+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:14.428 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:14.422+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:14.460 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:14.460 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:14.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:14.568 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:14.566+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:14.629 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:14.626+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:14.648 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:14.648 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:14.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:14.730 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:14.725+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:14.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:14.831 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:14.826+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:14.832 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:14.832 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:14.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:14.897 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:14.913 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:14.914 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:14.932 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:14.929+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:14.933 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:14.933 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:14.949 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:14.949 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:14.967 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:14.968 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:14.988 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:14.988 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:15.009 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:15.009 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:15.022 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:15.023 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:15.031 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:15.031 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:15.032 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:15.029+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:15.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:15.134 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:15.129+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:15.156 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:15.156 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:15.156 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:07:15.157 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:07:15.157 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:15.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:15.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:07:15.508 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:15.505+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:15.605 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:15.602+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.694866+0000", 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.943043292999999, 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002559076, 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.642 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.694866+0000", 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.694866+0000", 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.694864+0000", 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.694901+0000", 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6803e-05 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.694928+0000", 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7250000000000002e-05 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.694949+0000", 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0769999999999999e-05 2024-09-13T09:07:15.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.695021+0000", 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2282999999999999e-05 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.695053+0000", 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1585999999999999e-05 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.697311+0000", 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022582460000000002 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.697425+0000", 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011401200000000001 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-13T09:07:15.647 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.696311+0000", 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.941598618, 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025770900000000002, 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.696311+0000", 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.696311+0000", 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.696309+0000", 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.696364+0000", 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4669000000000001e-05 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.696401+0000", 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7222999999999999e-05 2024-09-13T09:07:15.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.696415+0000", 2024-09-13T09:07:15.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3397000000000001e-05 2024-09-13T09:07:15.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.696566+0000", 2024-09-13T09:07:15.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015115199999999999 2024-09-13T09:07:15.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.696603+0000", 2024-09-13T09:07:15.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7706000000000001e-05 2024-09-13T09:07:15.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.698776+0000", 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021728730000000001 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.698888+0000", 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111383 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.653 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:34 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'16)", 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.700100+0000", 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.937808861000001, 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002744682, 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.700100+0000", 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.700100+0000", 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.700099+0000", 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.700110+0000", 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1167e-05 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.700136+0000", 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6407e-05 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.700149+0000", 2024-09-13T09:07:15.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2866e-05 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.700268+0000", 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011846899999999999 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.700309+0000", 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0893000000000003e-05 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.702712+0000", 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024037680000000001 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.702845+0000", 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000132559 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'17)", 2024-09-13T09:07:15.658 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.701668+0000", 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.936241642999999, 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025778179999999999, 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.701668+0000", 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.701668+0000", 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.701666+0000", 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.701720+0000", 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4446e-05 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.701749+0000", 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9037e-05 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.701763+0000", 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3593e-05 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.701867+0000", 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104249 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.701906+0000", 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8926e-05 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.704171+0000", 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022648600000000001 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.704245+0000", 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4410000000000001e-05 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:54 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.737984+0000", 2024-09-13T09:07:15.663 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.899924819999999, 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023707849999999998, 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.737984+0000", 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.737984+0000", 2024-09-13T09:07:15.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.737983+0000", 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.737996+0000", 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3637e-05 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.738020+0000", 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3730000000000001e-05 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.738033+0000", 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2659e-05 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.738146+0000", 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113033 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.738184+0000", 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8102999999999998e-05 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.740266+0000", 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020817600000000002 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.740355+0000", 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9445999999999997e-05 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.668 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.739004+0000", 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.898904848000001, 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023892119999999999, 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.739004+0000", 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.739004+0000", 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.739003+0000", 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.739089+0000", 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6216000000000003e-05 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.739115+0000", 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5868999999999999e-05 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.739127+0000", 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.205e-05 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.739272+0000", 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014534300000000001 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.739330+0000", 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7402999999999999e-05 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.741300+0000", 2024-09-13T09:07:15.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019697439999999998 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.741394+0000", 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3893000000000004e-05 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.764526+0000", 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.873383638, 2024-09-13T09:07:15.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023438550000000002, 2024-09-13T09:07:15.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.674 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.674 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.764526+0000", 2024-09-13T09:07:15.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.764526+0000", 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.764524+0000", 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.764536+0000", 2024-09-13T09:07:15.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2123000000000001e-05 2024-09-13T09:07:15.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.764561+0000", 2024-09-13T09:07:15.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5043000000000001e-05 2024-09-13T09:07:15.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.764576+0000", 2024-09-13T09:07:15.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4544e-05 2024-09-13T09:07:15.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.764763+0000", 2024-09-13T09:07:15.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018674499999999999 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.764800+0000", 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7373000000000003e-05 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.766803+0000", 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020033270000000001 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.766869+0000", 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6002999999999998e-05 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.790285+0000", 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.847624119999999, 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025652370000000002, 2024-09-13T09:07:15.680 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.790285+0000", 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.790285+0000", 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.790284+0000", 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.790296+0000", 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2123000000000001e-05 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.790318+0000", 2024-09-13T09:07:15.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2143000000000001e-05 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.790333+0000", 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.502e-05 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.790445+0000", 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112092 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.790495+0000", 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9487000000000002e-05 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.792796+0000", 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023014649999999999 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.792850+0000", 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.401e-05 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.791467+0000", 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.846442239000002, 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023543180000000002, 2024-09-13T09:07:15.685 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.791467+0000", 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.791467+0000", 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.791465+0000", 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.791582+0000", 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011636199999999999 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.791636+0000", 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4357000000000002e-05 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.791688+0000", 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2068999999999999e-05 2024-09-13T09:07:15.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.791841+0000", 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000153306 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.791880+0000", 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8402999999999998e-05 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.793759+0000", 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001879492 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.793821+0000", 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2019000000000004e-05 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:85 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'55, mlcod=16'50)", 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.792847+0000", 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.845062533, 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023033789999999999, 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.792847+0000", 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.792847+0000", 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.792845+0000", 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.792861+0000", 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5726000000000002e-05 2024-09-13T09:07:15.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.792892+0000", 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.116e-05 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.792913+0000", 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0910000000000001e-05 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.793013+0000", 2024-09-13T09:07:15.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9889000000000005e-05 2024-09-13T09:07:15.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.793050+0000", 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6842999999999999e-05 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.795093+0000", 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020430040000000002 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.795150+0000", 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7469999999999997e-05 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.695 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:86 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.794368+0000", 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.843541428000002, 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026182890000000002, 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.794368+0000", 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.794368+0000", 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.794366+0000", 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:15.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.794381+0000", 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.524e-05 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.794417+0000", 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6053000000000002e-05 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.794436+0000", 2024-09-13T09:07:15.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8783e-05 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.794581+0000", 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014556200000000001 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.794620+0000", 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9069999999999997e-05 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.796917+0000", 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022968289999999998 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.796986+0000", 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8712000000000002e-05 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:89 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'59, mlcod=16'57)", 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.800900+0000", 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.837009552000001, 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025168410000000001, 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.800900+0000", 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.800900+0000", 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.800898+0000", 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.800990+0000", 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2305999999999993e-05 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.801016+0000", 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5483e-05 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.801028+0000", 2024-09-13T09:07:15.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1936999999999999e-05 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.801173+0000", 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014520900000000001 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.801211+0000", 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8633000000000001e-05 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.803321+0000", 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021098409999999999 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.803416+0000", 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5218999999999998e-05 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-13T09:07:15.705 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:56.965351+0000", 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.672558114000001, 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023707849999999998, 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.965351+0000", 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.965351+0000", 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.965349+0000", 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.965366+0000", 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6212999999999999e-05 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4443000000000002e-05 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.965423+0000", 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3357e-05 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.965525+0000", 2024-09-13T09:07:15.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010132900000000001 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.965565+0000", 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9962999999999997e-05 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.967662+0000", 2024-09-13T09:07:15.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020970009999999998 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:56.967722+0000", 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0198999999999997e-05 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'89)", 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:57.465454+0000", 2024-09-13T09:07:15.710 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.172455059000001, 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024760559999999999, 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.465454+0000", 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.465454+0000", 2024-09-13T09:07:15.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.465453+0000", 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.465533+0000", 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0073000000000004e-05 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.465548+0000", 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5007999999999999e-05 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.465555+0000", 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8560000000000001e-06 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.465644+0000", 2024-09-13T09:07:15.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9043999999999997e-05 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.465698+0000", 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3647000000000003e-05 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.467845+0000", 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002146893 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:57.467930+0000", 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5321999999999994e-05 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:154 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:58.235203+0000", 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.402706259999999, 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0065614820000000004, 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.715 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235203+0000", 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235203+0000", 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235202+0000", 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235217+0000", 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5033e-05 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235249+0000", 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1704e-05 2024-09-13T09:07:15.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235269+0000", 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0823e-05 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235602+0000", 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033258400000000003 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235669+0000", 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6809000000000004e-05 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.241606+0000", 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005937604 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.241764+0000", 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000158002 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.720 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:155 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=16'105)", 2024-09-13T09:07:15.720 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:58.235944+0000", 2024-09-13T09:07:15.720 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.401965096000001, 2024-09-13T09:07:15.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058962980000000003, 2024-09-13T09:07:15.720 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.720 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.720 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235944+0000", 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235944+0000", 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.235943+0000", 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236178+0000", 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000234925 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236212+0000", 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3623e-05 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236231+0000", 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9885999999999999e-05 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236835+0000", 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00060317599999999997 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236919+0000", 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3959e-05 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.241773+0000", 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048548649999999999 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.241840+0000", 2024-09-13T09:07:15.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6959999999999996e-05 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:156 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'110, mlcod=16'105)", 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:58.236702+0000", 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.401207475, 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0052596589999999999, 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236702+0000", 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236702+0000", 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236700+0000", 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236720+0000", 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9910000000000001e-05 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236744+0000", 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3759000000000001e-05 2024-09-13T09:07:15.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.236771+0000", 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7820000000000001e-05 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.238176+0000", 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014045360000000001 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.238236+0000", 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0566000000000003e-05 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.241851+0000", 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003614496 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.241961+0000", 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110395 2024-09-13T09:07:15.729 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:157 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'111, mlcod=16'105)", 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:58.237181+0000", 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.400727795000002, 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048477950000000002, 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237181+0000", 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237181+0000", 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237180+0000", 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:15.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237193+0000", 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.359e-05 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237242+0000", 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8668999999999998e-05 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237259+0000", 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6807e-05 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.239220+0000", 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019610909999999999 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.239278+0000", 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8103000000000003e-05 2024-09-13T09:07:15.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.241968+0000", 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002689819 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.242029+0000", 2024-09-13T09:07:15.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1205999999999996e-05 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:158 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'112, mlcod=16'105)", 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:58.237782+0000", 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.400127004000002, 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0053503190000000001, 2024-09-13T09:07:15.735 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237782+0000", 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237782+0000", 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237781+0000", 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237920+0000", 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013967299999999999 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237947+0000", 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6475999999999999e-05 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.237963+0000", 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6606999999999999e-05 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.240294+0000", 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023308949999999999 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.240349+0000", 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4969999999999997e-05 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.243062+0000", 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027125280000000001 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.243132+0000", 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0759999999999993e-05 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4207.0:159 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'113, mlcod=17'106)", 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:06:58.238782+0000", 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.399126761000002, 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050943869999999997, 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:15.740 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.238782+0000", 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.238782+0000", 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.238781+0000", 2024-09-13T09:07:15.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.238811+0000", 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0338999999999999e-05 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.238835+0000", 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4003e-05 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.238848+0000", 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2677000000000001e-05 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.242105+0000", 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032572679999999998 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.242156+0000", 2024-09-13T09:07:15.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0923e-05 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.243783+0000", 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001627568 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:06:58.243877+0000", 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3472000000000002e-05 2024-09-13T09:07:15.744 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.745 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.745 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.745 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:15.745 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:15.745 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:15.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:15.747 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:15.746+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:15.848 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:15.845+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:15.855 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:07:15.923 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:15.923 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:15.923 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:15.923 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:15.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:15.984 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:15.982+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:16.049 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:16.045+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:16.056 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:16.056 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:16.057 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:07:16.057 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:07:16.057 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:16.071 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:16.149 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:16.145+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:16.245 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:16.245 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:16.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:16.245 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:07:16.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.245 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4235.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:05.135667+0000", 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 11.108660156999999, 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024452839999999998, 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135667+0000", 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135667+0000", 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135667+0000", 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135675+0000", 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7059999999999993e-06 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135704+0000", 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8557000000000001e-05 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135723+0000", 2024-09-13T09:07:16.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.912e-05 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135849+0000", 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012599200000000001 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135918+0000", 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8949999999999995e-05 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.138025+0000", 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002107317 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.138113+0000", 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7138999999999994e-05 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:16.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4235.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:06.149048+0000", 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.095279802, 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017636500000000001, 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:16.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149048+0000", 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149048+0000", 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149047+0000", 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149056+0000", 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7499999999999992e-06 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149084+0000", 2024-09-13T09:07:16.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.815e-05 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149107+0000", 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2728999999999999e-05 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149237+0000", 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130433 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149279+0000", 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1425999999999999e-05 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.150740+0000", 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014616659999999999 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:16.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.150811+0000", 2024-09-13T09:07:16.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1025999999999994e-05 2024-09-13T09:07:16.256 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:16.256 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:16.256 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:16.256 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:16.256 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:16.256 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:16.257 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:16.253+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:16.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:16.358 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:16.354+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:16.379 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:16.379 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:16.380 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:16.380 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:16.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:16.484 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:16.482+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:16.560 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:16.561 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:16.561 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:07:16.561 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:07:16.561 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:16.561 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:16.558+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:16.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:16.661 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:16.657+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:16.740 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [] 2024-09-13T09:07:16.741 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:16.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:16.762 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:16.758+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:16.864 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:16.858+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:16.877 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:16.877 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:16.877 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:16.877 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:16.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:16.964 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:16.961+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:17.017 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:17.017 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:17.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:17.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:17.071 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:17.069+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:17.167 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:17.162+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:17.186 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4388.0:10 3.1c e25/22 3:3831950c:::rbd_directory:head v 25'1, mlcod=0'0)", 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:15.869065+0000", 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.318301457, 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089652299999999998, 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869065+0000", 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869065+0000", 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869065+0000", 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869072+0000", 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6809999999999998e-06 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869103+0000", 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0793999999999997e-05 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869113+0000", 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0013000000000001e-05 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869196+0000", 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2700000000000004e-05 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869215+0000", 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9009e-05 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869917+0000", 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070161400000000004 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:15.869962+0000", 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5090999999999999e-05 2024-09-13T09:07:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:17.192 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:17.201 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:17.218 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:17.219 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:17.236 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:17.236 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:17.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:17.264 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:17.267 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:17.262+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:17.292 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:17.292 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:17.311 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:17.311 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:17.324 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:17.324 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:17.325 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:17.343 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:17.344 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:17.363 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:17.364 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:17.368 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:17.366+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:17.383 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:17.383 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:17.469 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:17.466+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:17.470 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:17.470 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:17.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:17.571 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:17.566+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:17.606 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:17.606 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:17.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:17.672 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:17.670+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:17.734 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:17.734 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:17.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:17.795 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:17.793+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:17.834 INFO:teuthology.orchestra.run.smithi031.stdout:skipping zero size read 2024-09-13T09:07:17.834 INFO:teuthology.orchestra.run.smithi031.stdout:skipping zero size read 2024-09-13T09:07:17.841 INFO:teuthology.orchestra.run.smithi031.stdout:6 write 0x25dcca5 thru 0x25e2172 (0x54ce bytes) 2024-09-13T09:07:17.847 INFO:teuthology.orchestra.run.smithi031.stdout:7 read 0x9b5bcc thru 0x9c3a52 (0xde87 bytes) 2024-09-13T09:07:17.847 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:17.863 INFO:teuthology.orchestra.run.smithi031.stdout:9 write 0xb3331b8 thru 0xb3363e1 (0x322a bytes) 2024-09-13T09:07:17.865 INFO:teuthology.orchestra.run.smithi031.stdout:10 writesame 0x8461167 thru 0x846b40d data_size 0x1(0xa2a7 bytes) 2024-09-13T09:07:17.870 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:17.870 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:17.874 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:17.869+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:17.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:07:17.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:17.894 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:17.894 INFO:teuthology.orchestra.run.smithi031.stdout:12 write 0xa062b1b thru 0xa06f126 (0xc60c bytes) 2024-09-13T09:07:17.975 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:17.973+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:18.005 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:18.005 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:18.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:18.076 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:18.074+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:18.103 INFO:teuthology.orchestra.run.smithi031.stdout:13 read 0x86908ec thru 0x8697af5 (0x720a bytes) 2024-09-13T09:07:18.105 INFO:teuthology.orchestra.run.smithi031.stdout:14 read 0x10a6721 thru 0x10afb3a (0x941a bytes) 2024-09-13T09:07:18.105 INFO:teuthology.orchestra.run.smithi031.stdout:15 read 0x2d3d7f3 thru 0x2d440ef (0x68fd bytes) 2024-09-13T09:07:18.115 INFO:teuthology.orchestra.run.smithi031.stdout:16 write 0xdb16360 thru 0xdb244f9 (0xe19a bytes) 2024-09-13T09:07:18.120 INFO:teuthology.orchestra.run.smithi031.stdout:18 punch from 0x3bb4066 to 0x3bc38ee, (0xf888 bytes) 2024-09-13T09:07:18.121 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xe05aac 2024-09-13T09:07:18.138 INFO:teuthology.orchestra.run.smithi031.stdout:19 trunc from 0xdb244fa to 0xe05aac 2024-09-13T09:07:18.158 INFO:teuthology.orchestra.run.smithi031.stdout:21 read 0xb69946 thru 0xb6bcfd (0x23b8 bytes) 2024-09-13T09:07:18.158 INFO:teuthology.orchestra.run.smithi031.stdout:22 read 0x9742f3 thru 0x976834 (0x2542 bytes) 2024-09-13T09:07:18.158 INFO:teuthology.orchestra.run.smithi031.stdout:25 read 0x4f74e9 thru 0x4fba2e (0x4546 bytes) 2024-09-13T09:07:18.158 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xe6015ca 2024-09-13T09:07:18.177 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:18.173+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:18.184 INFO:teuthology.orchestra.run.smithi031.stdout:26 trunc from 0xe05aac to 0xe6015ca 2024-09-13T09:07:18.187 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:18.187 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:18.192 INFO:teuthology.orchestra.run.smithi031.stdout:28 write 0x2e6f56c thru 0x2e71010 (0x1aa5 bytes) 2024-09-13T09:07:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:29 read 0x1f2f65e thru 0x1f3c207 (0xcbaa bytes) 2024-09-13T09:07:18.197 INFO:teuthology.orchestra.run.smithi031.stdout:31 write 0x8554868 thru 0x855aaba (0x6253 bytes) 2024-09-13T09:07:18.201 INFO:teuthology.orchestra.run.smithi031.stdout:32 read 0xc321b64 thru 0xc3230e0 (0x157d bytes) 2024-09-13T09:07:18.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:18.204 INFO:teuthology.orchestra.run.smithi031.stdout:33 trunc from 0xe6015ca to 0xce59a23 2024-09-13T09:07:18.210 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:18.211 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:18.211 INFO:teuthology.orchestra.run.smithi031.stdout:37 read 0x13fb11c thru 0x13fe8a4 (0x3789 bytes) 2024-09-13T09:07:18.211 INFO:teuthology.orchestra.run.smithi031.stdout:38 write 0x7a9ab2b thru 0x7aa02b2 (0x5788 bytes) 2024-09-13T09:07:18.216 INFO:teuthology.orchestra.run.smithi031.stdout:40 read 0x3204b9 thru 0x32f144 (0xec8c bytes) 2024-09-13T09:07:18.224 INFO:teuthology.orchestra.run.smithi031.stdout:41 write 0xe855378 thru 0xe85f5e9 (0xa272 bytes) 2024-09-13T09:07:18.233 INFO:teuthology.orchestra.run.smithi031.stdout:42 punch from 0xcf689b0 to 0xcf7108f, (0x86df bytes) 2024-09-13T09:07:18.233 INFO:teuthology.orchestra.run.smithi031.stdout:43 read 0xa2c5b51 thru 0xa2c61f7 (0x6a7 bytes) 2024-09-13T09:07:18.233 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:18.233 INFO:teuthology.orchestra.run.smithi031.stdout:45 write 0x3cc0def thru 0x3ccdd5b (0xcf6d bytes) 2024-09-13T09:07:18.237 INFO:teuthology.orchestra.run.smithi031.stdout:46 write 0x8dcbb7 thru 0x8e4f57 (0x83a1 bytes) 2024-09-13T09:07:18.240 INFO:teuthology.orchestra.run.smithi031.stdout:48 writesame 0x217a577 thru 0x2189e37 data_size 0x1(0xf8c1 bytes) 2024-09-13T09:07:18.278 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:18.274+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:18.297 INFO:teuthology.orchestra.run.smithi031.stdout:49 read 0x19fd5d9 thru 0x1a01226 (0x3c4e bytes) 2024-09-13T09:07:18.299 INFO:teuthology.orchestra.run.smithi031.stdout:50 read 0x4895e9a thru 0x48a477c (0xe8e3 bytes) 2024-09-13T09:07:18.299 INFO:teuthology.orchestra.run.smithi031.stdout:51 clone 1 order 19 su 131072 sc 9 2024-09-13T09:07:18.326 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:18.326 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:18.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:18.379 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:18.374+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:18.480 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:18.477+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:18.508 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:18.508 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:18.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:18.580 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:18.578+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:18.656 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:18.656 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:18.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:18.682 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:18.677+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:18.782 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:18.777+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:18.834 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:18.834 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:18.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:18.896 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:18.894+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:18.976 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:18.976 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:18.984 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:18.981+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:18.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:19.085 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:19.081+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:19.120 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:19.120 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:07:19.121 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:07:19.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:07:19.164 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:19.164 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:19.178 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:07:19.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:19.228 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:19.226+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:19.286 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:19.281+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:19.302 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:19.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:19.386 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:19.385+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:19.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:19.436 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:19.437 INFO:teuthology.orchestra.run.smithi031.stdout:55 read 0xa3d9ee0 thru 0xa3dccc2 (0x2de3 bytes) 2024-09-13T09:07:19.437 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:19.437 INFO:teuthology.orchestra.run.smithi031.stdout:57 write 0x11302c3 thru 0x113a638 (0xa376 bytes) 2024-09-13T09:07:19.444 INFO:teuthology.orchestra.run.smithi031.stdout:58 read 0x55d013b thru 0x55d1de0 (0x1ca6 bytes) 2024-09-13T09:07:19.444 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:19.445 INFO:teuthology.orchestra.run.smithi031.stdout:61 read 0xa76ce8b thru 0xa76f254 (0x23ca bytes) 2024-09-13T09:07:19.460 INFO:teuthology.orchestra.run.smithi031.stdout:62 trunc from 0xe85f5ea to 0x575840b 2024-09-13T09:07:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:19.485 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:19.485 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 26, 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T09:06:44.149006+0000", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T09:07:19.172085+0000", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T09:06:55.611487+0000", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T09:06:46.637496+0000", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5804032, 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T09:07:19.486 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T09:07:19.487 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:06:55.131330+0000", 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:07:19.488 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:07:19.489 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:07:19.490 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:07:19.491 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:07:19.492 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:07:19.493 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:02.743102+0000", 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:07:19.495 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:07:19.496 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-09-13T09:07:19.497 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:07:19.498 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:07:19.499 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9800000190734863, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9800000190734863, 2024-09-13T09:07:19.500 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 4.380000114440918, 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 4.380000114440918, 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:12.517158+0000", 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:07:19.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T09:07:19.502 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "26", 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:07:19.503 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 4, 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 26, 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:07:19.504 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:07:19.505 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.506 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:19.507 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "33c6fc7d-9828-4500-b103-b96d1cd7f420", 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:19.508 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.509 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:19.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T09:07:19.511 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T09:07:19.512 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6818/3291756747", 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6822/3291756747", 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6830/3291756747", 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6826/3291756747", 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "869ea3c5-8e6e-4b49-b70e-083f242a7aae", 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:19.516 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:19.517 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.518 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/533520566", 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/533520566", 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/533520566", 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/533520566", 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e144d496-76ee-46e0-bdc6-135c77128535", 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:19.519 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:19.520 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:19.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T09:07:19.522 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.523 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6819/2253604482", 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6823/2253604482", 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/2253604482", 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6827/2253604482", 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:19.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "696b9414-9c19-4984-a8ff-14548e46b357", 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:19.526 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:19.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/3367214693", 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/3367214693", 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/3367214693", 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/3367214693", 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:07:19.531 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ac58040d-9b54-42ca-b143-1816abb54816", 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6800", 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6801", 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:19.533 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6802", 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6803", 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6806", 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6807", 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6804", 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6805", 2024-09-13T09:07:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6801/3778074750", 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6803/3778074750", 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6807/3778074750", 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6805/3778074750", 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2", 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6808", 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6809", 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6811", 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6813", 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6819", 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6824", 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6815", 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6817", 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6809/1926603087", 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6813/1926603087", 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6824/1926603087", 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6817/1926603087", 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d", 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6820", 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6821", 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6823", 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6825", 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6830", 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6831", 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6827", 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6828", 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6821/766305500", 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6825/766305500", 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6831/766305500", 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6828/766305500", 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f36c332c-9497-47e8-b943-f6b6dcaebeb8", 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6810", 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6812", 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6814", 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6816", 2024-09-13T09:07:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6826", 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6829", 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6818", 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6822", 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6812/1032956039", 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6816/1032956039", 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6829/1032956039", 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6822/1032956039", 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.183436+0000", 2024-09-13T09:07:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.264080+0000", 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.253587+0000", 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.244429+0000", 2024-09-13T09:07:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.369458+0000", 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.158089+0000", 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.217295+0000", 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.332254+0000", 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T09:07:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 2, 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:19.565 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:19.566 INFO:teuthology.orchestra.run.smithi031.stdout:63 flatten 2024-09-13T09:07:19.567 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:19.568 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:19.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:19.569 INFO:tasks.thrashosds.thrasher:j is {'epoch': 26, 'fsid': 'fe16bf6d-4166-4417-80fd-1123617a33d3', 'created': '2024-09-13T09:06:44.149006+0000', 'modified': '2024-09-13T09:07:19.172085+0000', 'last_up_change': '2024-09-13T09:06:55.611487+0000', 'last_in_change': '2024-09-13T09:06:46.637496+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:06:55.131330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:07:02.743102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:07:12.517158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 26, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '33c6fc7d-9828-4500-b103-b96d1cd7f420', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6818', 'nonce': 3291756747}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6822', 'nonce': 3291756747}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6830', 'nonce': 3291756747}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6826', 'nonce': 3291756747}]}, 'public_addr': '172.21.15.31:6818/3291756747', 'cluster_addr': '172.21.15.31:6822/3291756747', 'heartbeat_back_addr': '172.21.15.31:6830/3291756747', 'heartbeat_front_addr': '172.21.15.31:6826/3291756747', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 533520566}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 533520566}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 533520566}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 533520566}]}, 'public_addr': '172.21.15.31:6801/533520566', 'cluster_addr': '172.21.15.31:6803/533520566', 'heartbeat_back_addr': '172.21.15.31:6807/533520566', 'heartbeat_front_addr': '172.21.15.31:6805/533520566', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e144d496-76ee-46e0-bdc6-135c77128535', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6817', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2253604482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6821', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2253604482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6829', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 2253604482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6825', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 2253604482}]}, 'public_addr': '172.21.15.31:6819/2253604482', 'cluster_addr': '172.21.15.31:6823/2253604482', 'heartbeat_back_addr': '172.21.15.31:6831/2253604482', 'heartbeat_front_addr': '172.21.15.31:6827/2253604482', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '696b9414-9c19-4984-a8ff-14548e46b357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 3367214693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 3367214693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 3367214693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 3367214693}]}, 'public_addr': '172.21.15.31:6809/3367214693', 'cluster_addr': '172.21.15.31:6811/3367214693', 'heartbeat_back_addr': '172.21.15.31:6815/3367214693', 'heartbeat_front_addr': '172.21.15.31:6813/3367214693', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ac58040d-9b54-42ca-b143-1816abb54816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6800', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6801', 'nonce': 3778074750}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 3778074750}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 3778074750}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 3778074750}]}, 'public_addr': '172.21.15.60:6801/3778074750', 'cluster_addr': '172.21.15.60:6803/3778074750', 'heartbeat_back_addr': '172.21.15.60:6807/3778074750', 'heartbeat_front_addr': '172.21.15.60:6805/3778074750', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1926603087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6811', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 1926603087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6819', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6824', 'nonce': 1926603087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6815', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 1926603087}]}, 'public_addr': '172.21.15.60:6809/1926603087', 'cluster_addr': '172.21.15.60:6813/1926603087', 'heartbeat_back_addr': '172.21.15.60:6824/1926603087', 'heartbeat_front_addr': '172.21.15.60:6817/1926603087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 766305500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6823', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 766305500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 766305500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6827', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6828', 'nonce': 766305500}]}, 'public_addr': '172.21.15.60:6821/766305500', 'cluster_addr': '172.21.15.60:6825/766305500', 'heartbeat_back_addr': '172.21.15.60:6831/766305500', 'heartbeat_front_addr': '172.21.15.60:6828/766305500', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f36c332c-9497-47e8-b943-f6b6dcaebeb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6812', 'nonce': 1032956039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6816', 'nonce': 1032956039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 1032956039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6822', 'nonce': 1032956039}]}, 'public_addr': '172.21.15.60:6812/1032956039', 'cluster_addr': '172.21.15.60:6816/1032956039', 'heartbeat_back_addr': '172.21.15.60:6829/1032956039', 'heartbeat_front_addr': '172.21.15.60:6822/1032956039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.183436+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.264080+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.253587+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.244429+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.369458+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.158089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.217295+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.332254+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:07:19.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:07:19.570 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:19.570 INFO:teuthology.orchestra.run.smithi031.stdout:65 write 0xafb020e thru 0xafb9363 (0x9156 bytes) 2024-09-13T09:07:19.571 INFO:teuthology.orchestra.run.smithi031.stdout:66 read 0x5539c70 thru 0x5544ca1 (0xb032 bytes) 2024-09-13T09:07:19.571 INFO:teuthology.orchestra.run.smithi031.stdout:67 write 0x8b3a501 thru 0x8b45650 (0xb150 bytes) 2024-09-13T09:07:19.571 INFO:teuthology.orchestra.run.smithi031.stdout:68 writesame 0xd2c4555 thru 0xd2cfb05 data_size 0x1(0xb5b1 bytes) 2024-09-13T09:07:19.571 INFO:teuthology.orchestra.run.smithi031.stdout:69 punch from 0xef47cd to 0xf0365a, (0xee8d bytes) 2024-09-13T09:07:19.571 INFO:teuthology.orchestra.run.smithi031.stdout:72 read 0x99f6a65 thru 0x99fa380 (0x391c bytes) 2024-09-13T09:07:19.571 INFO:teuthology.orchestra.run.smithi031.stdout:76 punch from 0xce498cb to 0xce4b83f, (0x1f74 bytes) 2024-09-13T09:07:19.571 INFO:teuthology.orchestra.run.smithi031.stdout:77 writesame 0x1dcb29 thru 0x1e8b41 data_size 0x1(0xc019 bytes) 2024-09-13T09:07:19.571 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:19.565+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:19.574 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:19.587 INFO:teuthology.orchestra.run.smithi031.stdout:78 write 0x58e83f4 thru 0x58f1180 (0x8d8d bytes) 2024-09-13T09:07:19.590 INFO:teuthology.orchestra.run.smithi031.stdout:80 clone 2 order 18 su 2048 sc 3 2024-09-13T09:07:19.596 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:19.596 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:19.612 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:19.612 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:19.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:19.629 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:19.647 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:19.647 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:19.667 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:19.667 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:19.667 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:19.662+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:19.686 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:19.687 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:19.696 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:19.696 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:19.696 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:19.696 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:19.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:19.711 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:19.712 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:19.771 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:19.769+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:19.869 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:19.866+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:19.872 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:19.873 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:07:19.883 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:19.883 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:19.883 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:07:19.883 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:07:19.883 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:19.890 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-13T09:07:19.174474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590452,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":150,"num_read_kb":126,"num_write":167,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239828,"kb_used_data":3148,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754734892,"statfs":{"total":773094113280,"available":772848529408,"internally_reserved":0,"allocated":3223552,"data_stored":1943480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":242208312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.333365"},"pg_stats":[{"pgid":"3.1f","version":"25'1","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.890416+0000","last_change":"2024-09-13T09:07:15.854509+0000","last_active":"2024-09-13T09:07:15.890416+0000","last_peered":"2024-09-13T09:07:15.890416+0000","last_clean":"2024-09-13T09:07:15.890416+0000","last_became_active":"2024-09-13T09:07:13.855191+0000","last_became_peered":"2024-09-13T09:07:13.855191+0000","last_unstale":"2024-09-13T09:07:15.890416+0000","last_undegraded":"2024-09-13T09:07:15.890416+0000","last_fullsized":"2024-09-13T09:07:15.890416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:45:12.901944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.853796+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:07:15.853796+0000","last_peered":"2024-09-13T09:07:15.853796+0000","last_clean":"2024-09-13T09:07:15.853796+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:07:15.853796+0000","last_undegraded":"2024-09-13T09:07:15.853796+0000","last_fullsized":"2024-09-13T09:07:15.853796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.879980+0000","last_change":"2024-09-13T09:07:15.879980+0000","last_active":"2024-09-13T09:07:15.879980+0000","last_peered":"2024-09-13T09:07:15.879980+0000","last_clean":"2024-09-13T09:07:15.879980+0000","last_became_active":"2024-09-13T09:07:13.856004+0000","last_became_peered":"2024-09-13T09:07:13.856004+0000","last_unstale":"2024-09-13T09:07:15.879980+0000","last_undegraded":"2024-09-13T09:07:15.879980+0000","last_fullsized":"2024-09-13T09:07:15.879980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:37.046343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1c","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.870170+0000","last_change":"2024-09-13T09:07:15.861297+0000","last_active":"2024-09-13T09:07:15.870170+0000","last_peered":"2024-09-13T09:07:15.870170+0000","last_clean":"2024-09-13T09:07:15.870170+0000","last_became_active":"2024-09-13T09:07:13.851127+0000","last_became_peered":"2024-09-13T09:07:13.851127+0000","last_unstale":"2024-09-13T09:07:15.870170+0000","last_undegraded":"2024-09-13T09:07:15.870170+0000","last_fullsized":"2024-09-13T09:07:15.870170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:00:14.601636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.214167+0000","last_change":"2024-09-13T09:07:13.863418+0000","last_active":"2024-09-13T09:07:15.214167+0000","last_peered":"2024-09-13T09:07:15.214167+0000","last_clean":"2024-09-13T09:07:15.214167+0000","last_became_active":"2024-09-13T09:07:13.862565+0000","last_became_peered":"2024-09-13T09:07:13.862565+0000","last_unstale":"2024-09-13T09:07:15.214167+0000","last_undegraded":"2024-09-13T09:07:15.214167+0000","last_fullsized":"2024-09-13T09:07:15.214167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:32.305423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"25'2","reported_seq":18,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.897634+0000","last_change":"2024-09-13T09:07:15.861376+0000","last_active":"2024-09-13T09:07:15.897634+0000","last_peered":"2024-09-13T09:07:15.897634+0000","last_clean":"2024-09-13T09:07:15.897634+0000","last_became_active":"2024-09-13T09:07:13.852916+0000","last_became_peered":"2024-09-13T09:07:13.852916+0000","last_unstale":"2024-09-13T09:07:15.897634+0000","last_undegraded":"2024-09-13T09:07:15.897634+0000","last_fullsized":"2024-09-13T09:07:15.897634+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:03.726697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.214196+0000","last_change":"2024-09-13T09:07:13.858875+0000","last_active":"2024-09-13T09:07:15.214196+0000","last_peered":"2024-09-13T09:07:15.214196+0000","last_clean":"2024-09-13T09:07:15.214196+0000","last_became_active":"2024-09-13T09:07:13.858058+0000","last_became_peered":"2024-09-13T09:07:13.858058+0000","last_unstale":"2024-09-13T09:07:15.214196+0000","last_undegraded":"2024-09-13T09:07:15.214196+0000","last_fullsized":"2024-09-13T09:07:15.214196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:27:37.829718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.854162+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:07:15.854162+0000","last_peered":"2024-09-13T09:07:15.854162+0000","last_clean":"2024-09-13T09:07:15.854162+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:07:15.854162+0000","last_undegraded":"2024-09-13T09:07:15.854162+0000","last_fullsized":"2024-09-13T09:07:15.854162+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.854461+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:07:15.854461+0000","last_peered":"2024-09-13T09:07:15.854461+0000","last_clean":"2024-09-13T09:07:15.854461+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:07:15.854461+0000","last_undegraded":"2024-09-13T09:07:15.854461+0000","last_fullsized":"2024-09-13T09:07:15.854461+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.859376+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:07:15.859376+0000","last_peered":"2024-09-13T09:07:15.859376+0000","last_clean":"2024-09-13T09:07:15.859376+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:07:15.859376+0000","last_undegraded":"2024-09-13T09:07:15.859376+0000","last_fullsized":"2024-09-13T09:07:15.859376+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.873331+0000","last_change":"2024-09-13T09:07:15.873331+0000","last_active":"2024-09-13T09:07:15.873331+0000","last_peered":"2024-09-13T09:07:15.873331+0000","last_clean":"2024-09-13T09:07:15.873331+0000","last_became_active":"2024-09-13T09:07:13.866813+0000","last_became_peered":"2024-09-13T09:07:13.866813+0000","last_unstale":"2024-09-13T09:07:15.873331+0000","last_undegraded":"2024-09-13T09:07:15.873331+0000","last_fullsized":"2024-09-13T09:07:15.873331+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.439602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.14","version":"25'1","reported_seq":16,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.885042+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:07:15.885042+0000","last_peered":"2024-09-13T09:07:15.885042+0000","last_clean":"2024-09-13T09:07:15.885042+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:07:15.885042+0000","last_undegraded":"2024-09-13T09:07:15.885042+0000","last_fullsized":"2024-09-13T09:07:15.885042+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":30,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.216139+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:07:15.216139+0000","last_peered":"2024-09-13T09:07:15.216139+0000","last_clean":"2024-09-13T09:07:15.216139+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:07:15.216139+0000","last_undegraded":"2024-09-13T09:07:15.216139+0000","last_fullsized":"2024-09-13T09:07:15.216139+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.214301+0000","last_change":"2024-09-13T09:07:13.859512+0000","last_active":"2024-09-13T09:07:15.214301+0000","last_peered":"2024-09-13T09:07:15.214301+0000","last_clean":"2024-09-13T09:07:15.214301+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:07:15.214301+0000","last_undegraded":"2024-09-13T09:07:15.214301+0000","last_fullsized":"2024-09-13T09:07:15.214301+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T09:07:13.851287+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:07:13.851287+0000","last_peered":"2024-09-13T09:07:13.851287+0000","last_clean":"2024-09-13T09:07:13.851287+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:07:13.851287+0000","last_undegraded":"2024-09-13T09:07:13.851287+0000","last_fullsized":"2024-09-13T09:07:13.851287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.214080+0000","last_change":"2024-09-13T09:07:13.865402+0000","last_active":"2024-09-13T09:07:15.214080+0000","last_peered":"2024-09-13T09:07:15.214080+0000","last_clean":"2024-09-13T09:07:15.214080+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:07:15.214080+0000","last_undegraded":"2024-09-13T09:07:15.214080+0000","last_fullsized":"2024-09-13T09:07:15.214080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.216354+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:07:15.216354+0000","last_peered":"2024-09-13T09:07:15.216354+0000","last_clean":"2024-09-13T09:07:15.216354+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:07:15.216354+0000","last_undegraded":"2024-09-13T09:07:15.216354+0000","last_fullsized":"2024-09-13T09:07:15.216354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T09:07:13.873025+0000","last_change":"2024-09-13T09:07:13.872020+0000","last_active":"2024-09-13T09:07:13.873025+0000","last_peered":"2024-09-13T09:07:13.873025+0000","last_clean":"2024-09-13T09:07:13.873025+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:07:13.873025+0000","last_undegraded":"2024-09-13T09:07:13.873025+0000","last_fullsized":"2024-09-13T09:07:13.873025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:14.845591+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:07:14.845591+0000","last_peered":"2024-09-13T09:07:14.845591+0000","last_clean":"2024-09-13T09:07:14.845591+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:07:14.845591+0000","last_undegraded":"2024-09-13T09:07:14.845591+0000","last_fullsized":"2024-09-13T09:07:14.845591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.859774+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:07:15.859774+0000","last_peered":"2024-09-13T09:07:15.859774+0000","last_clean":"2024-09-13T09:07:15.859774+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:07:15.859774+0000","last_undegraded":"2024-09-13T09:07:15.859774+0000","last_fullsized":"2024-09-13T09:07:15.859774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.873298+0000","last_change":"2024-09-13T09:07:15.873298+0000","last_active":"2024-09-13T09:07:15.873298+0000","last_peered":"2024-09-13T09:07:15.873298+0000","last_clean":"2024-09-13T09:07:15.873298+0000","last_became_active":"2024-09-13T09:07:13.852418+0000","last_became_peered":"2024-09-13T09:07:13.852418+0000","last_unstale":"2024-09-13T09:07:15.873298+0000","last_undegraded":"2024-09-13T09:07:15.873298+0000","last_fullsized":"2024-09-13T09:07:15.873298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:58.473601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"21'2","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.215977+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:07:15.215977+0000","last_peered":"2024-09-13T09:07:15.215977+0000","last_clean":"2024-09-13T09:07:15.215977+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:07:15.215977+0000","last_undegraded":"2024-09-13T09:07:15.215977+0000","last_fullsized":"2024-09-13T09:07:15.215977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.853678+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:07:15.853678+0000","last_peered":"2024-09-13T09:07:15.853678+0000","last_clean":"2024-09-13T09:07:15.853678+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:07:15.853678+0000","last_undegraded":"2024-09-13T09:07:15.853678+0000","last_fullsized":"2024-09-13T09:07:15.853678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.215393+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:07:15.215393+0000","last_peered":"2024-09-13T09:07:15.215393+0000","last_clean":"2024-09-13T09:07:15.215393+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:07:15.215393+0000","last_undegraded":"2024-09-13T09:07:15.215393+0000","last_fullsized":"2024-09-13T09:07:15.215393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T09:07:13.870226+0000","last_change":"2024-09-13T09:07:13.865569+0000","last_active":"2024-09-13T09:07:13.870226+0000","last_peered":"2024-09-13T09:07:13.870226+0000","last_clean":"2024-09-13T09:07:13.870226+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:07:13.870226+0000","last_undegraded":"2024-09-13T09:07:13.870226+0000","last_fullsized":"2024-09-13T09:07:13.870226+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.215668+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:07:15.215668+0000","last_peered":"2024-09-13T09:07:15.215668+0000","last_clean":"2024-09-13T09:07:15.215668+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:07:15.215668+0000","last_undegraded":"2024-09-13T09:07:15.215668+0000","last_fullsized":"2024-09-13T09:07:15.215668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.214195+0000","last_change":"2024-09-13T09:07:13.861341+0000","last_active":"2024-09-13T09:07:15.214195+0000","last_peered":"2024-09-13T09:07:15.214195+0000","last_clean":"2024-09-13T09:07:15.214195+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:07:15.214195+0000","last_undegraded":"2024-09-13T09:07:15.214195+0000","last_fullsized":"2024-09-13T09:07:15.214195+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.215988+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:07:15.215988+0000","last_peered":"2024-09-13T09:07:15.215988+0000","last_clean":"2024-09-13T09:07:15.215988+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:07:15.215988+0000","last_undegraded":"2024-09-13T09:07:15.215988+0000","last_fullsized":"2024-09-13T09:07:15.215988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:14.848493+0000","last_change":"2024-09-13T09:06:56.632919+0000","last_active":"2024-09-13T09:07:14.848493+0000","last_peered":"2024-09-13T09:07:14.848493+0000","last_clean":"2024-09-13T09:07:14.848493+0000","last_became_active":"2024-09-13T09:06:56.630117+0000","last_became_peered":"2024-09-13T09:06:56.630117+0000","last_unstale":"2024-09-13T09:07:14.848493+0000","last_undegraded":"2024-09-13T09:07:14.848493+0000","last_fullsized":"2024-09-13T09:07:14.848493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:32.893066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.214220+0000","last_change":"2024-09-13T09:07:13.864733+0000","last_active":"2024-09-13T09:07:15.214220+0000","last_peered":"2024-09-13T09:07:15.214220+0000","last_clean":"2024-09-13T09:07:15.214220+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:07:15.214220+0000","last_undegraded":"2024-09-13T09:07:15.214220+0000","last_fullsized":"2024-09-13T09:07:15.214220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.879561+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:07:15.879561+0000","last_peered":"2024-09-13T09:07:15.879561+0000","last_clean":"2024-09-13T09:07:15.879561+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:07:15.879561+0000","last_undegraded":"2024-09-13T09:07:15.879561+0000","last_fullsized":"2024-09-13T09:07:15.879561+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.880308+0000","last_change":"2024-09-13T09:07:15.880308+0000","last_active":"2024-09-13T09:07:15.880308+0000","last_peered":"2024-09-13T09:07:15.880308+0000","last_clean":"2024-09-13T09:07:15.880308+0000","last_became_active":"2024-09-13T09:07:13.858870+0000","last_became_peered":"2024-09-13T09:07:13.858870+0000","last_unstale":"2024-09-13T09:07:15.880308+0000","last_undegraded":"2024-09-13T09:07:15.880308+0000","last_fullsized":"2024-09-13T09:07:15.880308+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:07.836035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:16.095554+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:07:16.095554+0000","last_peered":"2024-09-13T09:07:16.095554+0000","last_clean":"2024-09-13T09:07:16.095554+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:07:16.095554+0000","last_undegraded":"2024-09-13T09:07:16.095554+0000","last_fullsized":"2024-09-13T09:07:16.095554+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.873583+0000","last_change":"2024-09-13T09:07:15.873583+0000","last_active":"2024-09-13T09:07:15.873583+0000","last_peered":"2024-09-13T09:07:15.873583+0000","last_clean":"2024-09-13T09:07:15.873583+0000","last_became_active":"2024-09-13T09:07:13.859769+0000","last_became_peered":"2024-09-13T09:07:13.859769+0000","last_unstale":"2024-09-13T09:07:15.873583+0000","last_undegraded":"2024-09-13T09:07:15.873583+0000","last_fullsized":"2024-09-13T09:07:15.873583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:52.161129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.880473+0000","last_change":"2024-09-13T09:07:15.880473+0000","last_active":"2024-09-13T09:07:15.880473+0000","last_peered":"2024-09-13T09:07:15.880473+0000","last_clean":"2024-09-13T09:07:15.880473+0000","last_became_active":"2024-09-13T09:07:13.854213+0000","last_became_peered":"2024-09-13T09:07:13.854213+0000","last_unstale":"2024-09-13T09:07:15.880473+0000","last_undegraded":"2024-09-13T09:07:15.880473+0000","last_fullsized":"2024-09-13T09:07:15.880473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:29:21.568604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.859429+0000","last_change":"2024-09-13T09:07:15.859429+0000","last_active":"2024-09-13T09:07:15.859429+0000","last_peered":"2024-09-13T09:07:15.859429+0000","last_clean":"2024-09-13T09:07:15.859429+0000","last_became_active":"2024-09-13T09:07:13.855153+0000","last_became_peered":"2024-09-13T09:07:13.855153+0000","last_unstale":"2024-09-13T09:07:15.859429+0000","last_undegraded":"2024-09-13T09:07:15.859429+0000","last_fullsized":"2024-09-13T09:07:15.859429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:15.261153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.873663+0000","last_change":"2024-09-13T09:07:15.873663+0000","last_active":"2024-09-13T09:07:15.873663+0000","last_peered":"2024-09-13T09:07:15.873663+0000","last_clean":"2024-09-13T09:07:15.873663+0000","last_became_active":"2024-09-13T09:07:13.849277+0000","last_became_peered":"2024-09-13T09:07:13.849277+0000","last_unstale":"2024-09-13T09:07:15.873663+0000","last_undegraded":"2024-09-13T09:07:15.873663+0000","last_fullsized":"2024-09-13T09:07:15.873663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:46:42.059444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.860752+0000","last_change":"2024-09-13T09:07:15.860752+0000","last_active":"2024-09-13T09:07:15.860752+0000","last_peered":"2024-09-13T09:07:15.860752+0000","last_clean":"2024-09-13T09:07:15.860752+0000","last_became_active":"2024-09-13T09:07:13.865003+0000","last_became_peered":"2024-09-13T09:07:13.865003+0000","last_unstale":"2024-09-13T09:07:15.860752+0000","last_undegraded":"2024-09-13T09:07:15.860752+0000","last_fullsized":"2024-09-13T09:07:15.860752+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:17:26.840113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.11","version":"25'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.899611+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:07:15.899611+0000","last_peered":"2024-09-13T09:07:15.899611+0000","last_clean":"2024-09-13T09:07:15.899611+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:07:15.899611+0000","last_undegraded":"2024-09-13T09:07:15.899611+0000","last_fullsized":"2024-09-13T09:07:15.899611+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":25,"num_read_kb":22,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T09:07:13.870175+0000","last_change":"2024-09-13T09:07:13.863745+0000","last_active":"2024-09-13T09:07:13.870175+0000","last_peered":"2024-09-13T09:07:13.870175+0000","last_clean":"2024-09-13T09:07:13.870175+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:07:13.870175+0000","last_undegraded":"2024-09-13T09:07:13.870175+0000","last_fullsized":"2024-09-13T09:07:13.870175+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T09:07:15.880613+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:07:15.880613+0000","last_peered":"2024-09-13T09:07:15.880613+0000","last_clean":"2024-09-13T09:07:15.880613+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:07:15.880613+0000","last_undegraded":"2024-09-13T09:07:15.880613+0000","last_fullsized":"2024-09-13T09:07:15.880613+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":65,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":26,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":245760,"data_stored":88867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4199999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":88886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":88867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":843776,"data_stored":679243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":88867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.742}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":237568,"data_stored":82775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":233472,"data_stored":82756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":921600,"data_stored":743219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":30,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":35,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:07:19.892 INFO:tasks.thrashosds.thrasher:Setting 3.1c to [0, 1, 7, 6] 2024-09-13T09:07:19.893 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1c', '0', '1', '7', '6'] 2024-09-13T09:07:19.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1c 0 1 7 6 2024-09-13T09:07:19.897 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:19.988 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:19.986+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:20.066 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4235.0:5 2.2 e20/18 2:5cea7035:::rbd_info:head v 20'1, mlcod=0'0)", 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:05.135667+0000", 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.930689227, 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024452839999999998, 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:20.067 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135667+0000", 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135667+0000", 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135667+0000", 2024-09-13T09:07:20.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135675+0000", 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7059999999999993e-06 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135704+0000", 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8557000000000001e-05 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135723+0000", 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.912e-05 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135849+0000", 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012599200000000001 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.135918+0000", 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8949999999999995e-05 2024-09-13T09:07:20.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.138025+0000", 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002107317 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:05.138113+0000", 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7138999999999994e-05 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:20.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4235.0:7 2.2 e21/18 2:5cea7035:::rbd_info:head v 21'2, mlcod=0'0)", 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:06.149048+0000", 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 13.917308872, 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017636500000000001, 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149048+0000", 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149048+0000", 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149047+0000", 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149056+0000", 2024-09-13T09:07:20.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7499999999999992e-06 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149084+0000", 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.815e-05 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149107+0000", 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2728999999999999e-05 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:20.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149237+0000", 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130433 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.149279+0000", 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1425999999999999e-05 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.150740+0000", 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014616659999999999 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:06.150811+0000", 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1025999999999994e-05 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:20.076 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:20.077 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:20.073+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:20.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:20.178 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:20.174+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:20.256 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:20.256 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:20.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:20.279 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:20.277+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:20.380 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:20.377+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:20.440 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:20.440 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:20.441 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:07:20.441 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:07:20.441 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:20.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:20.476 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-13T09:07:20.481 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:20.477+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:20.580 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:20.580 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:20.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:20.580 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:07:20.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.580 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:509 3.0 e27/22 3:04479396:::rbd_id.image_client.0-clone2:head v 27'3, mlcod=26'1)", 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:20.250948+0000", 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.32824114399999998, 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00090565700000000003, 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250948+0000", 2024-09-13T09:07:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250948+0000", 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250948+0000", 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:20.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250956+0000", 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5730000000000008e-06 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250978+0000", 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2073e-05 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250992+0000", 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.414e-05 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.251054+0000", 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1390999999999995e-05 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:20.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.251077+0000", 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3439e-05 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.251823+0000", 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000745461 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.251854+0000", 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1115999999999998e-05 2024-09-13T09:07:20.585 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:20.586 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:20.586 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:20.586 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:20.586 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:20.586 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:20.590 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:20.586+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:20.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:20.688 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:20.686+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:20.726 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:20.727 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:20.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:20.791 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:20.789+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:20.879 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:20.880 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:20.889 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:20.885+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:20.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:20.989 INFO:teuthology.orchestra.run.smithi031.stdout:81 trunc from 0xd2cfb06 to 0xb09d9f2 2024-09-13T09:07:20.990 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:20.989+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:21.022 INFO:teuthology.orchestra.run.smithi031.stdout:82 write 0x3f064d thru 0x3fd1cd (0xcb81 bytes) 2024-09-13T09:07:21.049 INFO:teuthology.orchestra.run.smithi031.stdout:84 write 0x65ee5c3 thru 0x65fa612 (0xc050 bytes) 2024-09-13T09:07:21.059 INFO:teuthology.orchestra.run.smithi031.stdout:86 writesame 0x7258167 thru 0x7266ae0 data_size 0x5(0xe97a bytes) 2024-09-13T09:07:21.076 INFO:teuthology.orchestra.run.smithi031.stdout:87 punch from 0x9e7b0fe to 0x9e7cf43, (0x1e45 bytes) 2024-09-13T09:07:21.077 INFO:teuthology.orchestra.run.smithi031.stdout:88 read 0xa8b6118 thru 0xa8b7bbb (0x1aa4 bytes) 2024-09-13T09:07:21.077 INFO:teuthology.orchestra.run.smithi031.stdout:90 punch from 0x8db612 to 0x8e1b65, (0x6553 bytes) 2024-09-13T09:07:21.077 INFO:teuthology.orchestra.run.smithi031.stdout:91 read 0x6b8d9cd thru 0x6b97565 (0x9b99 bytes) 2024-09-13T09:07:21.077 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:21.077 INFO:teuthology.orchestra.run.smithi031.stdout:93 write 0x4932187 thru 0x4936551 (0x43cb bytes) 2024-09-13T09:07:21.084 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:21.084 INFO:teuthology.orchestra.run.smithi031.stdout:98 read 0x13aea0c thru 0x13b4338 (0x592d bytes) 2024-09-13T09:07:21.084 INFO:teuthology.orchestra.run.smithi031.stdout:99 read 0x64460fc thru 0x644ab4d (0x4a52 bytes) 2024-09-13T09:07:21.085 INFO:teuthology.orchestra.run.smithi031.stdout:102 punch from 0x8c676a4 to 0x8c7722b, (0xfb87 bytes) 2024-09-13T09:07:21.085 INFO:teuthology.orchestra.run.smithi031.stdout:103 write 0x81cd1b7 thru 0x81d1711 (0x455b bytes) 2024-09-13T09:07:21.088 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:21.088 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:21.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:21.088 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:21.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:696 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 139264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997590+0000", 2024-09-13T09:07:21.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088926232999999993, 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077878929999999997, 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 696 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997590+0000", 2024-09-13T09:07:21.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997590+0000", 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997589+0000", 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997597+0000", 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6529999999999994e-06 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997603+0000", 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7089999999999999e-06 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997606+0000", 2024-09-13T09:07:21.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2899999999999998e-06 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005226+0000", 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007620028 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005296+0000", 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9811e-05 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005378+0000", 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1866000000000005e-05 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:697 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 145408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997609+0000", 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088906956999999995, 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079351600000000001, 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 697 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997609+0000", 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997609+0000", 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997609+0000", 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997610+0000", 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.313e-06 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997613+0000", 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0699999999999998e-06 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997616+0000", 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005444+0000", 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00782827 2024-09-13T09:07:21.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005492+0000", 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7763999999999999e-05 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005544+0000", 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2524000000000001e-05 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:698 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 151552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997623+0000", 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088892935000000006, 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081589620000000005, 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 698 2024-09-13T09:07:21.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997623+0000", 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997623+0000", 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997623+0000", 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997624+0000", 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.361e-06 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997627+0000", 2024-09-13T09:07:21.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4120000000000001e-06 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997633+0000", 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8119999999999996e-06 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005587+0000", 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079541890000000004 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005676+0000", 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8385000000000002e-05 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005782+0000", 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106135 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:699 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 157696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997636+0000", 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088880417000000003, 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084346319999999992, 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 699 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997636+0000", 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997636+0000", 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997635+0000", 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997637+0000", 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2550000000000001e-06 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997640+0000", 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1489999999999998e-06 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997642+0000", 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4530000000000001e-06 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005862+0000", 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082194480000000007 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.005964+0000", 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101891 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006070+0000", 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010665700000000001 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:700 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997644+0000", 2024-09-13T09:07:21.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088871681999999994, 2024-09-13T09:07:21.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085826370000000006, 2024-09-13T09:07:21.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 700 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997644+0000", 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997644+0000", 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997644+0000", 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997646+0000", 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.623e-06 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997649+0000", 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999998e-06 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997651+0000", 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3690000000000001e-06 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006111+0000", 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084597739999999998 2024-09-13T09:07:21.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006165+0000", 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4007e-05 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006227+0000", 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2090999999999999e-05 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:701 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 169984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997665+0000", 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088850728000000004, 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087740060000000009, 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 701 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997665+0000", 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997665+0000", 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997665+0000", 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997667+0000", 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997675+0000", 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.148e-06 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997680+0000", 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006265+0000", 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085851639999999993 2024-09-13T09:07:21.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006354+0000", 2024-09-13T09:07:21.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8428000000000001e-05 2024-09-13T09:07:21.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006439+0000", 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5657999999999999e-05 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:702 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 176128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997685+0000", 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088830714000000005, 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088979790000000003, 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 702 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997685+0000", 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997685+0000", 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997685+0000", 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997687+0000", 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.148e-06 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997692+0000", 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5620000000000003e-06 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997697+0000", 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2189999999999998e-06 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006485+0000", 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008787988 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006530+0000", 2024-09-13T09:07:21.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5852000000000001e-05 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006583+0000", 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2751999999999999e-05 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:703 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 182272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997700+0000", 2024-09-13T09:07:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088815669999999999, 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090068649999999993, 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 703 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997700+0000", 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997700+0000", 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997700+0000", 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997702+0000", 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9649999999999998e-06 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997708+0000", 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997713+0000", 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5929999999999999e-06 2024-09-13T09:07:21.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006616+0000", 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008902709 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006663+0000", 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7138000000000002e-05 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006707+0000", 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4233999999999997e-05 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:704 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997717+0000", 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088799285000000006, 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009108517, 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 704 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997717+0000", 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997717+0000", 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997716+0000", 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997718+0000", 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.866e-06 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997724+0000", 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-06 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997737+0000", 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3294e-05 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006738+0000", 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090013019999999992 2024-09-13T09:07:21.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006781+0000", 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2843000000000003e-05 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006825+0000", 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3946000000000002e-05 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:705 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997745+0000", 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088771349999999999, 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092053359999999997, 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 705 2024-09-13T09:07:21.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997745+0000", 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997745+0000", 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997744+0000", 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997747+0000", 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6460000000000002e-06 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997753+0000", 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-06 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997758+0000", 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3030000000000001e-06 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006855+0000", 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090971750000000007 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006906+0000", 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1248000000000001e-05 2024-09-13T09:07:21.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006950+0000", 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3763000000000001e-05 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:706 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 200704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997762+0000", 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088753911000000005, 2024-09-13T09:07:21.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093043029999999999, 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 706 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997762+0000", 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997762+0000", 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997762+0000", 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997764+0000", 2024-09-13T09:07:21.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:07:21.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997771+0000", 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997776+0000", 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8849999999999998e-06 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006981+0000", 2024-09-13T09:07:21.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092050229999999997 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007024+0000", 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3009000000000001e-05 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007066+0000", 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2781999999999998e-05 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:707 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 206848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997780+0000", 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088736323000000006, 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094079739999999995, 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 707 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997780+0000", 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997780+0000", 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997779+0000", 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997781+0000", 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9089999999999998e-06 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997787+0000", 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3689999999999998e-06 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997792+0000", 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.955e-06 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007098+0000", 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093065379999999996 2024-09-13T09:07:21.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007144+0000", 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6276999999999998e-05 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007188+0000", 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3205999999999999e-05 2024-09-13T09:07:21.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:708 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997795+0000", 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088720707999999995, 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009507415, 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 708 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997795+0000", 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997795+0000", 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997795+0000", 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997805+0000", 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.662e-06 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997812+0000", 2024-09-13T09:07:21.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9750000000000001e-06 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997817+0000", 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007218+0000", 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094007610000000005 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007258+0000", 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0453e-05 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007303+0000", 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4245000000000001e-05 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:709 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997841+0000", 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088674702999999994, 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095819149999999999, 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 709 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997841+0000", 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997841+0000", 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997841+0000", 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997844+0000", 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997850+0000", 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.708e-06 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997855+0000", 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8409999999999999e-06 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007335+0000", 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094794200000000006 2024-09-13T09:07:21.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007379+0000", 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4656000000000001e-05 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007423+0000", 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3834000000000001e-05 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:710 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997860+0000", 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088656291999999998, 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096747900000000008, 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 710 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997860+0000", 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997860+0000", 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997859+0000", 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997862+0000", 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997868+0000", 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9939999999999997e-06 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997872+0000", 2024-09-13T09:07:21.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.968e-06 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007453+0000", 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095814290000000007 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007493+0000", 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0391e-05 2024-09-13T09:07:21.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007534+0000", 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0989999999999999e-05 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:711 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997876+0000", 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088640135999999994, 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098663190000000001, 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 711 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997876+0000", 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997876+0000", 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997875+0000", 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997878+0000", 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2510000000000001e-06 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997883+0000", 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997888+0000", 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7910000000000004e-06 2024-09-13T09:07:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007568+0000", 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096808020000000005 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007698+0000", 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013000899999999999 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007742+0000", 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3720000000000002e-05 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:712 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997892+0000", 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088623508000000004, 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010014729, 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 712 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997892+0000", 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997892+0000", 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997892+0000", 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997894+0000", 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:07:21.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997900+0000", 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9909999999999999e-06 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997905+0000", 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007778+0000", 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098734119999999998 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007827+0000", 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8983999999999997e-05 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007907+0000", 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9887000000000003e-05 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:713 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997913+0000", 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088603115999999996, 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01025295, 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 713 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997913+0000", 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997913+0000", 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997912+0000", 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997914+0000", 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.114e-06 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997920+0000", 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5450000000000003e-06 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997932+0000", 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2146e-05 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007957+0000", 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024358000000001 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008072+0000", 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115931 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008166+0000", 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3356000000000005e-05 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:714 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 249856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997945+0000", 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088571163999999994, 2024-09-13T09:07:21.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010338592000000001, 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 714 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997945+0000", 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997945+0000", 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997944+0000", 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997947+0000", 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3719999999999999e-06 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997951+0000", 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997954+0000", 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7480000000000002e-06 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008199+0000", 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010245687 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008240+0000", 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0729999999999998e-05 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008283+0000", 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3424000000000003e-05 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:715 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997956+0000", 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.088560246999999995, 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010529686, 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 715 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997956+0000", 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997956+0000", 2024-09-13T09:07:21.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997955+0000", 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997957+0000", 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-06 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997960+0000", 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.982e-06 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997963+0000", 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-06 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008313+0000", 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010349975000000001 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008398+0000", 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5719999999999999e-05 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008485+0000", 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6916999999999995e-05 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.186 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:21.186 INFO:teuthology.orchestra.run.smithi031.stdout:104 writesame 0x1c3cfe0 thru 0x1c41d56 data_size 0x1(0x4d77 bytes) 2024-09-13T09:07:21.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:21.188 INFO:teuthology.orchestra.run.smithi031.stdout:105 read 0x40ddda3 thru 0x40e3f61 (0x61bf bytes) 2024-09-13T09:07:21.188 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:21.189 INFO:teuthology.orchestra.run.smithi031.stdout:108 writesame 0xa574200 thru 0xa579037 data_size 0x1(0x4e38 bytes) 2024-09-13T09:07:21.189 INFO:teuthology.orchestra.run.smithi031.stdout:109 trunc from 0xb09d9f2 to 0x4a8d60f 2024-09-13T09:07:21.189 INFO:teuthology.orchestra.run.smithi031.stdout:112 read 0x3556795 thru 0x355edb9 (0x8625 bytes) 2024-09-13T09:07:21.189 INFO:teuthology.orchestra.run.smithi031.stdout:115 punch from 0x1757a4b to 0x1758b72, (0x1127 bytes) 2024-09-13T09:07:21.189 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:21.185+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:21.193 INFO:teuthology.orchestra.run.smithi031.stdout:116 write 0xaf7f944 thru 0xaf8e6d0 (0xed8d bytes) 2024-09-13T09:07:21.199 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.1c pg_upmap_items mapping to [0->1,7->6] 2024-09-13T09:07:21.205 INFO:teuthology.orchestra.run.smithi031.stdout:117 read 0x4a6b0fa thru 0x4a78b6c (0xda73 bytes) 2024-09-13T09:07:21.207 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:07:21.207 INFO:teuthology.orchestra.run.smithi031.stdout:119 writesame 0x1365842 thru 0x136f0e4 data_size 0x19(0x98a3 bytes) 2024-09-13T09:07:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:120 write 0x4ee8b9d thru 0x4eef82d (0x6c91 bytes) 2024-09-13T09:07:21.225 INFO:teuthology.orchestra.run.smithi031.stdout:122 read 0x6b9b706 thru 0x6ba24a6 (0x6da1 bytes) 2024-09-13T09:07:21.225 INFO:teuthology.orchestra.run.smithi031.stdout:123 writesame 0x7ba107e thru 0x7ba2311 data_size 0x4(0x1294 bytes) 2024-09-13T09:07:21.232 INFO:teuthology.orchestra.run.smithi031.stdout:125 read 0x69bfc25 thru 0x69c3f2a (0x4306 bytes) 2024-09-13T09:07:21.232 INFO:teuthology.orchestra.run.smithi031.stdout:126 punch from 0x2b214aa to 0x2b29ae0, (0x8636 bytes) 2024-09-13T09:07:21.232 INFO:teuthology.orchestra.run.smithi031.stdout:127 read 0x874de69 thru 0x8758999 (0xab31 bytes) 2024-09-13T09:07:21.234 INFO:teuthology.orchestra.run.smithi031.stdout:128 read 0x94144c3 thru 0x9414e7b (0x9b9 bytes) 2024-09-13T09:07:21.234 INFO:teuthology.orchestra.run.smithi031.stdout:132 write 0x98f9f2c thru 0x9909a0f (0xfae4 bytes) 2024-09-13T09:07:21.254 INFO:teuthology.orchestra.run.smithi031.stdout:133 write 0xb90ad04 thru 0xb9185df (0xd8dc bytes) 2024-09-13T09:07:21.262 INFO:teuthology.orchestra.run.smithi031.stdout:134 punch from 0x3c41ce1 to 0x3c45f89, (0x42a8 bytes) 2024-09-13T09:07:21.265 INFO:teuthology.orchestra.run.smithi031.stdout:136 trunc from 0xb9185e0 to 0x97fe800 2024-09-13T09:07:21.286 INFO:teuthology.orchestra.run.smithi031.stdout:137 read 0x5a86eb1 thru 0x5a92bde (0xbd2e bytes) 2024-09-13T09:07:21.286 INFO:teuthology.orchestra.run.smithi031.stdout:138 read 0x3572c11 thru 0x357a514 (0x7904 bytes) 2024-09-13T09:07:21.286 INFO:teuthology.orchestra.run.smithi031.stdout:139 read 0x94b0058 thru 0x94b1324 (0x12cd bytes) 2024-09-13T09:07:21.286 INFO:teuthology.orchestra.run.smithi031.stdout:140 read 0x7f58fef thru 0x7f5fa91 (0x6aa3 bytes) 2024-09-13T09:07:21.286 INFO:teuthology.orchestra.run.smithi031.stdout:141 punch from 0x513672 to 0x513bd7, (0x565 bytes) 2024-09-13T09:07:21.288 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:21.282+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:21.297 INFO:teuthology.orchestra.run.smithi031.stdout:143 writesame 0xb7e3299 thru 0xb7e5986 data_size 0x21(0x26ee bytes) 2024-09-13T09:07:21.297 INFO:teuthology.orchestra.run.smithi031.stdout:145 read 0xa31dc10 thru 0xa328cd6 (0xb0c7 bytes) 2024-09-13T09:07:21.297 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:21.297 INFO:teuthology.orchestra.run.smithi031.stdout:148 writesame 0x7d33e35 thru 0x7d3cbce data_size 0x1d(0x8d9a bytes) 2024-09-13T09:07:21.307 INFO:teuthology.orchestra.run.smithi031.stdout:151 write 0x5eb0712 thru 0x5ebb209 (0xaaf8 bytes) 2024-09-13T09:07:21.315 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:21.315 INFO:teuthology.orchestra.run.smithi031.stdout:155 write 0x368a5ad thru 0x368c555 (0x1fa9 bytes) 2024-09-13T09:07:21.324 INFO:teuthology.orchestra.run.smithi031.stdout:156 read 0x9cc01bd thru 0x9ccbd29 (0xbb6d bytes) 2024-09-13T09:07:21.334 INFO:teuthology.orchestra.run.smithi031.stdout:157 write 0xca2debd thru 0xca3da3c (0xfb80 bytes) 2024-09-13T09:07:21.340 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:1064 3.10 3:0fb30186:::rbd_data.1124f3df56b6.0000000000000329:head [set-alloc-hint object_size 262144 write_size 262144,write 149504~21053 in=21053b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.340005+0000", 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0012730179999999999, 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001296731, 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for sub ops", 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1064 2024-09-13T09:07:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340005+0000", 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340005+0000", 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340003+0000", 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340038+0000", 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4409999999999998e-05 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340057+0000", 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8893e-05 2024-09-13T09:07:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340066+0000", 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6039999999999999e-06 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340129+0000", 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3025000000000001e-05 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340285+0000", 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015562799999999999 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.340450+0000", 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016486699999999999 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-13T09:07:21.345 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:21.346 INFO:teuthology.orchestra.run.smithi031.stdout:158 write 0x2cf37af thru 0x2cf9891 (0x60e3 bytes) 2024-09-13T09:07:21.350 INFO:teuthology.orchestra.run.smithi031.stdout:159 write 0x8ecc77c thru 0x8ed36be (0x6f43 bytes) 2024-09-13T09:07:21.357 INFO:teuthology.orchestra.run.smithi031.stdout:160 read 0xa1b2984 thru 0xa1c1823 (0xeea0 bytes) 2024-09-13T09:07:21.357 INFO:teuthology.orchestra.run.smithi031.stdout:163 punch from 0x352cdfb to 0x353a01b, (0xd220 bytes) 2024-09-13T09:07:21.358 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:21.358 INFO:teuthology.orchestra.run.smithi031.stdout:165 punch from 0xc4d69f to 0xc58c55, (0xb5b6 bytes) 2024-09-13T09:07:21.358 INFO:teuthology.orchestra.run.smithi031.stdout:167 write 0x4a1fc0f thru 0x4a256ed (0x5adf bytes) 2024-09-13T09:07:21.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:21.366 INFO:teuthology.orchestra.run.smithi031.stdout:168 write 0xb900564 thru 0xb90b95b (0xb3f8 bytes) 2024-09-13T09:07:21.374 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:21.374 INFO:teuthology.orchestra.run.smithi031.stdout:171 writesame 0x7a93c9c thru 0x7aa3b3d data_size 0xb(0xfea2 bytes) 2024-09-13T09:07:21.388 INFO:teuthology.orchestra.run.smithi031.stdout:172 trunc from 0xca3da3d to 0xa31fa89 2024-09-13T09:07:21.388 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:21.386+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:21.406 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:21.415 INFO:teuthology.orchestra.run.smithi031.stdout:174 writesame 0xb76b334 thru 0xb76ee2a data_size 0x1(0x3af7 bytes) 2024-09-13T09:07:21.417 INFO:teuthology.orchestra.run.smithi031.stdout:175 trunc from 0xb76ee2b to 0xa54688b 2024-09-13T09:07:21.425 INFO:teuthology.orchestra.run.smithi031.stdout:176 read 0x3cc7e4c thru 0x3cd4f2b (0xd0e0 bytes) 2024-09-13T09:07:21.430 INFO:teuthology.orchestra.run.smithi031.stdout:177 read 0xa0c7f66 thru 0xa0c9e2b (0x1ec6 bytes) 2024-09-13T09:07:21.431 INFO:teuthology.orchestra.run.smithi031.stdout:178 write 0x6a83fed thru 0x6a9112e (0xd142 bytes) 2024-09-13T09:07:21.437 INFO:teuthology.orchestra.run.smithi031.stdout:181 read 0x1303dad thru 0x130ac1c (0x6e70 bytes) 2024-09-13T09:07:21.438 INFO:teuthology.orchestra.run.smithi031.stdout:183 punch from 0x589d1c8 to 0x58a4149, (0x6f81 bytes) 2024-09-13T09:07:21.438 INFO:teuthology.orchestra.run.smithi031.stdout:186 write 0x290441c thru 0x2906f87 (0x2b6c bytes) 2024-09-13T09:07:21.446 INFO:teuthology.orchestra.run.smithi031.stdout:189 write 0x6d2270f thru 0x6d28f26 (0x6818 bytes) 2024-09-13T09:07:21.454 INFO:teuthology.orchestra.run.smithi031.stdout:190 punch from 0xccc680 to 0xcd729f, (0xac1f bytes) 2024-09-13T09:07:21.463 INFO:teuthology.orchestra.run.smithi031.stdout:191 trunc from 0xa54688b to 0x4581c54 2024-09-13T09:07:21.489 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:21.486+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:21.508 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:21.509 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:21.509 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:21.509 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:21.509 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:21.517 INFO:teuthology.orchestra.run.smithi031.stdout:192 write 0x9f8354f thru 0x9f9317f (0xfc31 bytes) 2024-09-13T09:07:21.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:21.527 INFO:teuthology.orchestra.run.smithi031.stdout:194 read 0x4a35691 thru 0x4a3696f (0x12df bytes) 2024-09-13T09:07:21.527 INFO:teuthology.orchestra.run.smithi031.stdout:195 read 0x2b3bd51 thru 0x2b41818 (0x5ac8 bytes) 2024-09-13T09:07:21.527 INFO:teuthology.orchestra.run.smithi031.stdout:197 clone 3 order 20 su 262144 sc 14 2024-09-13T09:07:21.590 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:21.585+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:21.644 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.946839397000002, 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840076, 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 34 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:21.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694022+0000", 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694033+0000", 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.116e-05 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694051+0000", 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694065+0000", 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3716e-05 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700273+0000", 2024-09-13T09:07:21.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062082320000000002 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700405+0000", 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131416 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700608+0000", 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020368099999999999 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705556+0000", 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049475760000000004 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705654+0000", 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8444999999999997e-05 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705741+0000", 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6416999999999996e-05 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705863+0000", 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121876 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.946771559999998, 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:21.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.946463546, 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:21.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:21.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.946375632999999, 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:21.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:21.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:21.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:21.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675851271999999, 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:21.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:21.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:21.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675763508999999, 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:21.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:21.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675711455999998, 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:21.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:21.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675526374, 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:21.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:21.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:21.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675451885000001, 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:21.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:21.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675397565000001, 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.67519948, 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.675118217000001, 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:21.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:21.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:21.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.459525+0000", 2024-09-13T09:07:21.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.181337138, 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011570051, 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 134 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459525+0000", 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459525+0000", 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459524+0000", 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459809+0000", 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028482799999999999 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459837+0000", 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459854+0000", 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7269999999999999e-05 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.465588+0000", 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057341859999999996 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.465720+0000", 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013131899999999999 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.465939+0000", 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021951199999999999 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.470868+0000", 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049289449999999997 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.470964+0000", 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6273000000000005e-05 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471039+0000", 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4899000000000002e-05 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471095+0000", 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5569999999999998e-05 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.459875+0000", 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.180986830999998, 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011494964999999999, 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 135 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459875+0000", 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459875+0000", 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459874+0000", 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:21.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459886+0000", 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.253e-05 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459910+0000", 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.37e-05 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459927+0000", 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7244e-05 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.467010+0000", 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070830650000000004 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.467126+0000", 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011595 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.467315+0000", 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189161 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471193+0000", 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038778300000000001 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471213+0000", 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9293e-05 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471290+0000", 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7269999999999997e-05 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471370+0000", 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0246000000000004e-05 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.459944+0000", 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.180917834999999, 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011686119999999999, 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 136 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459944+0000", 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459944+0000", 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459943+0000", 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459955+0000", 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1616e-05 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459974+0000", 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.942e-05 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.459995+0000", 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0576000000000001e-05 2024-09-13T09:07:21.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.468337+0000", 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083425730000000007 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.468455+0000", 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118126 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.468676+0000", 2024-09-13T09:07:21.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000220335 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471394+0000", 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027179690000000002 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471488+0000", 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4889000000000006e-05 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471551+0000", 2024-09-13T09:07:21.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2998999999999998e-05 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471630+0000", 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8733000000000007e-05 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.180823501999999, 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:21.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:21.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:21.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:21.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:21.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.180507864999999, 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.747 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.180447534999999, 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:21.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.180389629, 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:21.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:21.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:21.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.180087031999999, 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:21.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:21.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:21.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:21.765 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:21.766 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:21.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:21.767 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:21.762+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:21.868 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:21.865+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:21.943 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:21.943 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:21.943 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:21.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:21.977 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:21.978 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:21.978 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:21.973+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:21.994 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:22.010 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:22.011 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:22.027 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:22.027 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:22.047 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:22.048 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:22.068 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:22.069 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:22.066+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:22.088 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:22.088 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:22.095 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:22.095 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:22.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:22.110 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:22.110 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:22.170 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:22.166+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:22.216 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:07:22.235 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:22.236 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:22.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:22.299 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:22.297+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:22.311 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone2 intact 2024-09-13T09:07:22.371 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:22.369+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:22.420 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:22.420 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:22.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:22.472 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:22.470+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:22.476 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-13T09:07:22.573 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:22.570+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:22.629 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:22.629 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:22.648 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:22.695 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:22.693+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:22.770 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:22.771 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:22.775 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:22.770+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:22.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:22.876 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:22.874+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:22.920 INFO:teuthology.orchestra.run.smithi031.stdout:198 write 0x280a37f thru 0x28159a6 (0xb628 bytes) 2024-09-13T09:07:22.921 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:22.921 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:22.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:22.938 INFO:teuthology.orchestra.run.smithi031.stdout:199 write 0x7d5d100 thru 0x7d6600e (0x8f0f bytes) 2024-09-13T09:07:22.944 INFO:teuthology.orchestra.run.smithi031.stdout:201 read 0x7e99e3b thru 0x7ea6b8b (0xcd51 bytes) 2024-09-13T09:07:22.956 INFO:teuthology.orchestra.run.smithi031.stdout:202 write 0xaf35666 thru 0xaf36363 (0xcfe bytes) 2024-09-13T09:07:22.960 INFO:teuthology.orchestra.run.smithi031.stdout:204 read 0x4f62ca9 thru 0x4f7135e (0xe6b6 bytes) 2024-09-13T09:07:22.961 INFO:teuthology.orchestra.run.smithi031.stdout:206 writesame 0x18a3ee9 thru 0x18ae18b data_size 0xb(0xa2a3 bytes) 2024-09-13T09:07:22.980 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:22.978+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:22.988 INFO:teuthology.orchestra.run.smithi031.stdout:207 trunc from 0xaf36364 to 0x44cccfd 2024-09-13T09:07:23.078 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:23.074+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:23.092 INFO:teuthology.orchestra.run.smithi031.stdout:209 write 0x99770cb thru 0x9984abd (0xd9f3 bytes) 2024-09-13T09:07:23.101 INFO:teuthology.orchestra.run.smithi031.stdout:210 write 0x52cd42c thru 0x52d810e (0xace3 bytes) 2024-09-13T09:07:23.109 INFO:teuthology.orchestra.run.smithi031.stdout:211 writesame 0x76a2051 thru 0x76a62cf data_size 0x1(0x427f bytes) 2024-09-13T09:07:23.115 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:23.115 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:23.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:23.152 INFO:teuthology.orchestra.run.smithi031.stdout:213 write 0xa265a09 thru 0xa274b61 (0xf159 bytes) 2024-09-13T09:07:23.179 INFO:teuthology.orchestra.run.smithi031.stdout:214 write 0xc962ea6 thru 0xc966a9c (0x3bf7 bytes) 2024-09-13T09:07:23.185 INFO:teuthology.orchestra.run.smithi031.stdout:215 read 0x2693013 thru 0x2699562 (0x6550 bytes) 2024-09-13T09:07:23.185 INFO:teuthology.orchestra.run.smithi031.stdout:216 punch from 0x33ad002 to 0x33b41bc, (0x71ba bytes) 2024-09-13T09:07:23.186 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:23.186 INFO:teuthology.orchestra.run.smithi031.stdout:218 read 0x88274be thru 0x882d747 (0x628a bytes) 2024-09-13T09:07:23.186 INFO:teuthology.orchestra.run.smithi031.stdout:220 read 0xaefa53b thru 0xaf07fd1 (0xda97 bytes) 2024-09-13T09:07:23.186 INFO:teuthology.orchestra.run.smithi031.stdout:222 read 0x626b31b thru 0x627458b (0x9271 bytes) 2024-09-13T09:07:23.188 INFO:teuthology.orchestra.run.smithi031.stdout:223 trunc from 0xc966a9d to 0xd2b000e 2024-09-13T09:07:23.199 INFO:teuthology.orchestra.run.smithi031.stdout:224 read 0x2891584 thru 0x28a101a (0xfa97 bytes) 2024-09-13T09:07:23.202 INFO:teuthology.orchestra.run.smithi031.stdout:225 read 0x9037cd1 thru 0x90399b3 (0x1ce3 bytes) 2024-09-13T09:07:23.202 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:23.202 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:23.204 INFO:teuthology.orchestra.run.smithi031.stdout:229 trunc from 0xd2b000e to 0xc3ad28c 2024-09-13T09:07:23.220 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:23.218+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:23.225 INFO:teuthology.orchestra.run.smithi031.stdout:230 punch from 0x82a11a1 to 0x82a3162, (0x1fc1 bytes) 2024-09-13T09:07:23.231 INFO:teuthology.orchestra.run.smithi031.stdout:231 trunc from 0xc3ad28c to 0x9e30413 2024-09-13T09:07:23.245 INFO:teuthology.orchestra.run.smithi031.stdout:232 write 0x2def503 thru 0x2df7c7b (0x8779 bytes) 2024-09-13T09:07:23.274 INFO:teuthology.orchestra.run.smithi031.stdout:234 write 0xbc0d20f thru 0xbc145ef (0x73e1 bytes) 2024-09-13T09:07:23.279 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:23.277+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:23.282 INFO:teuthology.orchestra.run.smithi031.stdout:235 read 0x70e4d2e thru 0x70ef125 (0xa3f8 bytes) 2024-09-13T09:07:23.282 INFO:teuthology.orchestra.run.smithi031.stdout:238 write 0x203b4f5 thru 0x203e7d3 (0x32df bytes) 2024-09-13T09:07:23.310 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:23.310 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:23.312 INFO:teuthology.orchestra.run.smithi031.stdout:239 writesame 0x7f235b6 thru 0x7f31f46 data_size 0x13(0xe991 bytes) 2024-09-13T09:07:23.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:23.347 INFO:teuthology.orchestra.run.smithi031.stdout:240 trunc from 0xbc145f0 to 0x35b435d 2024-09-13T09:07:23.412 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:23.410+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:23.434 INFO:teuthology.orchestra.run.smithi031.stdout:241 punch from 0x2c6db5f to 0x2c6f2bc, (0x175d bytes) 2024-09-13T09:07:23.434 INFO:teuthology.orchestra.run.smithi031.stdout:244 writesame 0xc83a44 thru 0xc8fd1a data_size 0x1(0xc2d7 bytes) 2024-09-13T09:07:23.482 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:23.478+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:23.501 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:23.501 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:23.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:23.523 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:23.523 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:23.523 INFO:teuthology.orchestra.run.smithi031.stdout:248 read 0x1cd696f thru 0x1cde519 (0x7bab bytes) 2024-09-13T09:07:23.524 INFO:teuthology.orchestra.run.smithi031.stdout:249 write 0x1f0695f thru 0x1f0d6ae (0x6d50 bytes) 2024-09-13T09:07:23.559 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:23.559 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:23.583 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:23.581+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:23.589 INFO:teuthology.orchestra.run.smithi031.stdout:252 writesame 0xa6020af thru 0xa60339d data_size 0x1(0x12ef bytes) 2024-09-13T09:07:23.598 INFO:teuthology.orchestra.run.smithi031.stdout:254 trunc from 0xa60339e to 0xdbe1958 2024-09-13T09:07:23.610 INFO:teuthology.orchestra.run.smithi031.stdout:255 writesame 0xcf9f03c thru 0xcfa1228 data_size 0xf(0x21ed bytes) 2024-09-13T09:07:23.618 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:23.618 INFO:teuthology.orchestra.run.smithi031.stdout:258 clone 4 order 25 su 131072 sc 15 2024-09-13T09:07:23.684 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:23.681+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:23.687 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:23.687 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:23.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:23.785 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:23.781+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:23.820 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:23.821 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:23.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:23.885 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:23.882+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:23.967 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:23.967 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:23.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:24.002 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone3 from 0xdbe1958 (overlap 0x35b435d) to 0x1ada1ae 2024-09-13T09:07:24.024 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:24.022+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:24.087 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:24.082+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:24.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:24.156 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:24.156 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:24.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:24.188 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:24.185+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:24.226 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-13T09:07:24.239 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:07:24.276 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:24.276 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:24.288 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:24.285+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:24.293 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:24.293 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:24.323 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:24.323 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:24.356 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:24.356 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:24.373 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:24.373 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:24.388 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:24.385+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:24.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:24.394 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:24.394 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:24.422 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:24.422 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:24.457 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:24.458 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:24.489 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:24.485+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:24.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:24.491 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:24.521 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:24.522 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:24.522 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:24.522 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:24.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:24.589 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:24.586+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:24.661 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:24.662 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:24.662 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:07:24.662 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:07:24.662 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:24.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:24.717 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:24.714+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:24.725 INFO:teuthology.orchestra.run.smithi031.stdout:260 write 0xd843212 thru 0xd84eea7 (0xbc96 bytes) 2024-09-13T09:07:24.767 INFO:teuthology.orchestra.run.smithi031.stdout:261 trunc from 0xdbe1958 to 0xc587fb9 2024-09-13T09:07:24.791 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:24.786+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:24.878 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:24.878 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:509 3.0 e27/22 3:04479396:::rbd_id.image_client.0-clone2:head v 27'3, mlcod=26'1)", 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:20.250948+0000", 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.6255790320000001, 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00090565700000000003, 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250948+0000", 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250948+0000", 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250948+0000", 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250956+0000", 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5730000000000008e-06 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250978+0000", 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2073e-05 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.250992+0000", 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.414e-05 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.251054+0000", 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1390999999999995e-05 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.251077+0000", 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3439e-05 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.251823+0000", 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000745461 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.251854+0000", 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1115999999999998e-05 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:600 3.b 3:d26d7aae:::rbd_data.1124702db1b3.000000000000000e:head [sparse-read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:24.883 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:20.590334+0000", 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.2861928650000003, 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039863399999999998, 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 600 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.590334+0000", 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.590334+0000", 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.590332+0000", 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.590346+0000", 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4313e-05 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.590369+0000", 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2892999999999999e-05 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.590384+0000", 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5270000000000001e-05 2024-09-13T09:07:24.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.590520+0000", 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013603899999999999 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:20.590733+0000", 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212532 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:721 3.6 e27/22 3:609ec655:::rbd_data.1124f3df56b6.00000000000002c3:head v 27'4, mlcod=25'2)", 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.011938+0000", 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.8645891450000001, 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016031769999999999, 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.011938+0000", 2024-09-13T09:07:24.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.011938+0000", 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.011936+0000", 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.011955+0000", 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.931e-05 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.011985+0000", 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9958999999999999e-05 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.011998+0000", 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3127e-05 2024-09-13T09:07:24.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.012133+0000", 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013457899999999999 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.012201+0000", 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8442999999999995e-05 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.013450+0000", 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001249153 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.013541+0000", 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0999000000000001e-05 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:870 3.0 e27/22 3:04608615:::rbd_data.1124f3df56b6.0000000000000197:head v 27'4, mlcod=26'2)", 2024-09-13T09:07:24.892 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.056781+0000", 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.8197456110000001, 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019374080000000001, 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056781+0000", 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056781+0000", 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056779+0000", 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056848+0000", 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9363000000000007e-05 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056891+0000", 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2806000000000003e-05 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056911+0000", 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0293e-05 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.057038+0000", 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012656200000000001 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.057105+0000", 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7377000000000002e-05 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.058627+0000", 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001521941 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.058719+0000", 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1586e-05 2024-09-13T09:07:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:873 3.c e27/22 3:32bef489:::rbd_data.1124f3df56b6.00000000000001c8:head v 27'3, mlcod=25'1)", 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.068091+0000", 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.8084363720000001, 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00216622, 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068091+0000", 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068091+0000", 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068090+0000", 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:24.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068122+0000", 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2410000000000003e-05 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068147+0000", 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4426e-05 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068179+0000", 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1956999999999999e-05 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068257+0000", 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8429000000000004e-05 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068294+0000", 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7200000000000003e-05 2024-09-13T09:07:24.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.070149+0000", 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001854252 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.070257+0000", 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108299 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:898 3.c e27/22 3:37582dcf:::rbd_data.1124f3df56b6.0000000000000294:head v 27'4, mlcod=25'2)", 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.117174+0000", 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.7593526009999998, 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001602448, 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.117174+0000", 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.117174+0000", 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.117172+0000", 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.117195+0000", 2024-09-13T09:07:24.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.117228+0000", 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2483000000000001e-05 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.117247+0000", 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.895e-05 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.117336+0000", 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9118999999999998e-05 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.117382+0000", 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6595999999999999e-05 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.118701+0000", 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0013189670000000001 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.118777+0000", 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5696000000000002e-05 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:905 3.c e27/22 3:32bef489:::rbd_data.1124f3df56b6.00000000000001c8:head v 27'5, mlcod=27'3)", 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.145074+0000", 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.7314534429999999, 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021751930000000002, 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.906 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145074+0000", 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145074+0000", 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145072+0000", 2024-09-13T09:07:24.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145130+0000", 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8075999999999998e-05 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145179+0000", 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8943000000000002e-05 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145199+0000", 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0395999999999999e-05 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145335+0000", 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013600900000000001 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145390+0000", 2024-09-13T09:07:24.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4639999999999999e-05 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.147144+0000", 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017538860000000001 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.147249+0000", 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105309 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:904 3.0 e27/22 3:04608615:::rbd_data.1124f3df56b6.0000000000000197:head v 27'5, mlcod=27'3)", 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.145256+0000", 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.731271531, 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018121599999999999, 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145256+0000", 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145256+0000", 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145253+0000", 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145269+0000", 2024-09-13T09:07:24.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6195999999999999e-05 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145298+0000", 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8816999999999999e-05 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145315+0000", 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7039999999999999e-05 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145431+0000", 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011561900000000001 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145487+0000", 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5686000000000002e-05 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146973+0000", 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014865220000000001 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.147068+0000", 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4523000000000003e-05 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:911 3.c e27/22 3:37582dcf:::rbd_data.1124f3df56b6.0000000000000294:head v 27'6, mlcod=27'3)", 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.146269+0000", 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.7302584850000002, 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018475690000000001, 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.915 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146269+0000", 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146269+0000", 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146266+0000", 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146285+0000", 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8830000000000001e-05 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146330+0000", 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4422999999999998e-05 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146352+0000", 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2736000000000001e-05 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146520+0000", 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016734599999999999 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.146583+0000", 2024-09-13T09:07:24.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3058999999999994e-05 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.148027+0000", 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001444346 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.148116+0000", 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9075999999999999e-05 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:916 3.6 e27/22 3:609ec655:::rbd_data.1124f3df56b6.00000000000002c3:head v 27'5, mlcod=26'3)", 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.150608+0000", 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.7259191650000001, 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001592514, 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.150608+0000", 2024-09-13T09:07:24.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.150608+0000", 2024-09-13T09:07:24.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.150605+0000", 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.150624+0000", 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8433000000000001e-05 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.150664+0000", 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0136000000000003e-05 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.150684+0000", 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9916999999999999e-05 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.150796+0000", 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112576 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.150860+0000", 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3829000000000002e-05 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.152087+0000", 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012267669999999999 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.152200+0000", 2024-09-13T09:07:24.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113416 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:923 3.c e27/22 3:32a44471:::rbd_data.1124f3df56b6.000000000000012b:head v 27'7, mlcod=27'5)", 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.161640+0000", 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.7148868639999999, 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015372490000000001, 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.161640+0000", 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.161640+0000", 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.161639+0000", 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.161652+0000", 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3677e-05 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.161680+0000", 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7662999999999999e-05 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.161694+0000", 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4286e-05 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.161842+0000", 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014751900000000001 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.161880+0000", 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7993e-05 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.163083+0000", 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001203104 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.163177+0000", 2024-09-13T09:07:24.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.467e-05 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:1074 3.c e28/22 3:35dadade:::rbd_data.1124f3df56b6.000000000000023a:head v 28'8, mlcod=27'6)", 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.354278+0000", 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.5222489299999999, 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001869735, 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354278+0000", 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354278+0000", 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354277+0000", 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:24.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354326+0000", 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8519000000000001e-05 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354350+0000", 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4426999999999998e-05 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354365+0000", 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4496e-05 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354492+0000", 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012715599999999999 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354540+0000", 2024-09-13T09:07:24.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7516999999999999e-05 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.356064+0000", 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015247310000000001 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.356148+0000", 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3559000000000004e-05 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:1081 3.c e28/22 3:333c41be:::rbd_data.1124f3df56b6.0000000000000127:head v 28'9, mlcod=27'7)", 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.362631+0000", 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.5138960319999999, 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001781253, 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.362631+0000", 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.362631+0000", 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.362629+0000", 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.362654+0000", 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4579999999999998e-05 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.362681+0000", 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7523000000000001e-05 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.362701+0000", 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9009999999999999e-05 2024-09-13T09:07:24.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.362787+0000", 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6248999999999994e-05 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.362821+0000", 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4455999999999997e-05 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.364293+0000", 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014722789999999999 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.364412+0000", 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118806 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:1169 3.b 3:d76e9a83:::rbd_data.1124f3df56b6.00000000000000a2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 181276~3044] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:07:24.939 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.443678+0000", 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.4328492939999999, 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032811250000000002, 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 1169 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443678+0000", 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443678+0000", 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443675+0000", 2024-09-13T09:07:24.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999971 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443693+0000", 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8318999999999999e-05 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443727+0000", 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4164e-05 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443743+0000", 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5423000000000001e-05 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443878+0000", 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013483200000000001 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.444184+0000", 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030661099999999998 2024-09-13T09:07:24.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.444549+0000", 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036525099999999998 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446190+0000", 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001640697 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446799+0000", 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00060851500000000001 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446897+0000", 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8202000000000006e-05 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446959+0000", 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2124000000000004e-05 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.4388.0:1169 3.b e28/22 ondisk, result = 0)", 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.446623+0000", 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.4299038039999998, 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035127100000000001, 2024-09-13T09:07:24.946 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446623+0000", 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446623+0000", 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446622+0000", 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446632+0000", 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0859999999999999e-05 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446659+0000", 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6883e-05 2024-09-13T09:07:24.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446675+0000", 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5279999999999999e-05 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446774+0000", 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9109000000000003e-05 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446789+0000", 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.575e-05 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446975+0000", 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185102 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:1183 3.c e28/22 3:333c41be:::rbd_data.1124f3df56b6.0000000000000127:head v 28'10, mlcod=28'8)", 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.467729+0000", 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.408798429, 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018442549999999999, 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.467729+0000", 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.467729+0000", 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.467726+0000", 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.467744+0000", 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7793999999999999e-05 2024-09-13T09:07:24.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.467789+0000", 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4558999999999998e-05 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.467808+0000", 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9253e-05 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.467913+0000", 2024-09-13T09:07:24.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104689 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.467967+0000", 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.418e-05 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.469491+0000", 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001523588 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.469573+0000", 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2366000000000004e-05 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:1187 3.c e28/22 3:32a44471:::rbd_data.1124f3df56b6.000000000000012b:head v 28'11, mlcod=28'8)", 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.468266+0000", 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.4082610529999999, 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001920798, 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.955 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468266+0000", 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468266+0000", 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468264+0000", 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:24.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468281+0000", 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7073000000000001e-05 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468318+0000", 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6396000000000002e-05 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468336+0000", 2024-09-13T09:07:24.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8099999999999999e-05 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468625+0000", 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028886500000000002 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468702+0000", 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6934999999999994e-05 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.470108+0000", 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014062429999999999 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.470187+0000", 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9012999999999997e-05 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:1209 3.c e28/22 3:35dadade:::rbd_data.1124f3df56b6.000000000000023a:head v 28'12, mlcod=28'10)", 2024-09-13T09:07:24.959 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.478740+0000", 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.3977866809999999, 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001774273, 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.478740+0000", 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:24.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.478740+0000", 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.478739+0000", 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.478751+0000", 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2624e-05 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.478778+0000", 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6713e-05 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.478815+0000", 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6893000000000001e-05 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.478941+0000", 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012572899999999999 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.478998+0000", 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7222999999999997e-05 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.480419+0000", 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014209789999999999 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.480515+0000", 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5768999999999997e-05 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.963 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:24.964 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:24.964 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:24.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:24.967 INFO:teuthology.orchestra.run.smithi031.stdout:262 read 0x1445870 thru 0x1445b0b (0x29c bytes) 2024-09-13T09:07:24.967 INFO:teuthology.orchestra.run.smithi031.stdout:263 read 0x142f0f2 thru 0x14368ca (0x77d9 bytes) 2024-09-13T09:07:24.967 INFO:teuthology.orchestra.run.smithi031.stdout:264 punch from 0x97ab88b to 0x97b912b, (0xd8a0 bytes) 2024-09-13T09:07:24.967 INFO:teuthology.orchestra.run.smithi031.stdout:265 writesame 0x57c4f40 thru 0x57cf06c data_size 0x1(0xa12d bytes) 2024-09-13T09:07:24.967 INFO:teuthology.orchestra.run.smithi031.stdout:266 writesame 0x821514b thru 0x8221acb data_size 0x5f(0xc981 bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:267 read 0x7a600e1 thru 0x7a635d1 (0x34f1 bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:268 punch from 0x9763726 to 0x976baf6, (0x83d0 bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:270 writesame 0x581e4e thru 0x585724 data_size 0x1(0x38d7 bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:271 write 0x8681ec1 thru 0x868b0a8 (0x91e8 bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:273 punch from 0x18c02f8 to 0x18c7a05, (0x770d bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:275 write 0x103edb9 thru 0x104abd5 (0xbe1d bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:276 punch from 0x669b7f6 to 0x66a7ac6, (0xc2d0 bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:277 punch from 0x7031556 to 0x703fd49, (0xe7f3 bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:280 read 0x949a548 thru 0x94a51a2 (0xac5b bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:281 write 0xb95bae5 thru 0xb95cf3e (0x145a bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:282 write 0xbfa6f1b thru 0xbfaad0b (0x3df1 bytes) 2024-09-13T09:07:24.968 INFO:teuthology.orchestra.run.smithi031.stdout:283 read 0x3240164 thru 0x3240d85 (0xc22 bytes) 2024-09-13T09:07:24.969 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:24.961+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:24.980 INFO:teuthology.orchestra.run.smithi031.stdout:284 trunc from 0xc587fb9 to 0xd907fc 2024-09-13T09:07:25.003 INFO:teuthology.orchestra.run.smithi031.stdout:285 read 0x465311 thru 0x46ec9b (0x998b bytes) 2024-09-13T09:07:25.026 INFO:teuthology.orchestra.run.smithi031.stdout:286 trunc from 0xd907fc to 0xb4d7b30 2024-09-13T09:07:25.036 INFO:teuthology.orchestra.run.smithi031.stdout:287 read 0x423b256 thru 0x4249a80 (0xe82b bytes) 2024-09-13T09:07:25.041 INFO:teuthology.orchestra.run.smithi031.stdout:288 trunc from 0xb4d7b30 to 0x8bda16a 2024-09-13T09:07:25.057 INFO:teuthology.orchestra.run.smithi031.stdout:289 write 0xa9ee48 thru 0xaa4e30 (0x5fe9 bytes) 2024-09-13T09:07:25.066 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:25.062+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:25.075 INFO:teuthology.orchestra.run.smithi031.stdout:290 writesame 0xb9e01f3 thru 0xb9e40e2 data_size 0x35(0x3ef0 bytes) 2024-09-13T09:07:25.078 INFO:teuthology.orchestra.run.smithi031.stdout:291 punch from 0x6f3eeb to 0x6f72c7, (0x33dc bytes) 2024-09-13T09:07:25.089 INFO:teuthology.orchestra.run.smithi031.stdout:292 write 0xc1bc655 thru 0xc1bd147 (0xaf3 bytes) 2024-09-13T09:07:25.103 INFO:teuthology.orchestra.run.smithi031.stdout:293 trunc from 0xc1bd148 to 0x4ca13f0 2024-09-13T09:07:25.124 INFO:teuthology.orchestra.run.smithi031.stdout:294 write 0x4336c35 thru 0x433fbd9 (0x8fa5 bytes) 2024-09-13T09:07:25.131 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:25.132 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:25.138 INFO:teuthology.orchestra.run.smithi031.stdout:295 writesame 0x4e76dc4 thru 0x4e84665 data_size 0x36(0xd8a2 bytes) 2024-09-13T09:07:25.140 INFO:teuthology.orchestra.run.smithi031.stdout:296 write 0xbf1aaf thru 0xbfab2f (0x9081 bytes) 2024-09-13T09:07:25.144 INFO:teuthology.orchestra.run.smithi031.stdout:297 read 0x3564c31 thru 0x35673eb (0x27bb bytes) 2024-09-13T09:07:25.145 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xec26668 2024-09-13T09:07:25.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:25.159 INFO:teuthology.orchestra.run.smithi031.stdout:298 trunc from 0x4e84666 to 0xec26668 2024-09-13T09:07:25.166 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:25.162+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:25.171 INFO:teuthology.orchestra.run.smithi031.stdout:304 read 0x5e897ab thru 0x5e90c3e (0x7494 bytes) 2024-09-13T09:07:25.171 INFO:teuthology.orchestra.run.smithi031.stdout:305 punch from 0x568081f to 0x5684577, (0x3d58 bytes) 2024-09-13T09:07:25.171 INFO:teuthology.orchestra.run.smithi031.stdout:306 write 0x30d7ae4 thru 0x30d7e18 (0x335 bytes) 2024-09-13T09:07:25.173 INFO:teuthology.orchestra.run.smithi031.stdout:308 read 0x26e1c08 thru 0x26e4546 (0x293f bytes) 2024-09-13T09:07:25.173 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.173 INFO:teuthology.orchestra.run.smithi031.stdout:311 read 0x79ae04f thru 0x79b2899 (0x484b bytes) 2024-09-13T09:07:25.174 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.175 INFO:teuthology.orchestra.run.smithi031.stdout:313 punch from 0xd524ba6 to 0xd527bbd, (0x3017 bytes) 2024-09-13T09:07:25.175 INFO:teuthology.orchestra.run.smithi031.stdout:315 read 0xe842eac thru 0xe84762e (0x4783 bytes) 2024-09-13T09:07:25.175 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.175 INFO:teuthology.orchestra.run.smithi031.stdout:317 writesame 0x44e6352 thru 0x44ee765 data_size 0x4(0x8414 bytes) 2024-09-13T09:07:25.184 INFO:teuthology.orchestra.run.smithi031.stdout:318 read 0x4663d51 thru 0x466dd7a (0xa02a bytes) 2024-09-13T09:07:25.185 INFO:teuthology.orchestra.run.smithi031.stdout:320 clone 5 order 23 su 4194304 sc 14 2024-09-13T09:07:25.210 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:07:25.283 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:25.280+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:25.290 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:25.290 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:25.290 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:25.290 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:25.290 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:25.306 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:25.342 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone4 from 0xec26668 (overlap 0xd907fc) to 0x0 2024-09-13T09:07:25.368 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:25.362+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:25.469 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:25.464+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:25.470 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:700 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 163840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997644+0000", 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4717764989999997, 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085826370000000006, 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 700 2024-09-13T09:07:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997644+0000", 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997644+0000", 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997644+0000", 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997646+0000", 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.623e-06 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997649+0000", 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999998e-06 2024-09-13T09:07:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997651+0000", 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3690000000000001e-06 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006111+0000", 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084597739999999998 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006165+0000", 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4007e-05 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006227+0000", 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2090999999999999e-05 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:701 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 169984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997665+0000", 2024-09-13T09:07:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4717555449999997, 2024-09-13T09:07:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0087740060000000009, 2024-09-13T09:07:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 701 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997665+0000", 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997665+0000", 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997665+0000", 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997667+0000", 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997675+0000", 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.148e-06 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997680+0000", 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006265+0000", 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085851639999999993 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006354+0000", 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8428000000000001e-05 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006439+0000", 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5657999999999999e-05 2024-09-13T09:07:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:702 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 176128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997685+0000", 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4717355310000002, 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088979790000000003, 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 702 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997685+0000", 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997685+0000", 2024-09-13T09:07:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997685+0000", 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997687+0000", 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.148e-06 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997692+0000", 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5620000000000003e-06 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997697+0000", 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2189999999999998e-06 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006485+0000", 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008787988 2024-09-13T09:07:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006530+0000", 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5852000000000001e-05 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006583+0000", 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2751999999999999e-05 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:703 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 182272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997700+0000", 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4717204869999998, 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090068649999999993, 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 703 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997700+0000", 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997700+0000", 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997700+0000", 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997702+0000", 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9649999999999998e-06 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997708+0000", 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5060000000000001e-06 2024-09-13T09:07:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997713+0000", 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5929999999999999e-06 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006616+0000", 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008902709 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006663+0000", 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7138000000000002e-05 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006707+0000", 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4233999999999997e-05 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:704 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 188416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997717+0000", 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4717041020000003, 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009108517, 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 704 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997717+0000", 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997717+0000", 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997716+0000", 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997718+0000", 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.866e-06 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997724+0000", 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5799999999999999e-06 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997737+0000", 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3294e-05 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006738+0000", 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090013019999999992 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006781+0000", 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2843000000000003e-05 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006825+0000", 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3946000000000002e-05 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:705 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 194560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997745+0000", 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.471676167, 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092053359999999997, 2024-09-13T09:07:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 705 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997745+0000", 2024-09-13T09:07:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997745+0000", 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997744+0000", 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997747+0000", 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6460000000000002e-06 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997753+0000", 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7100000000000001e-06 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997758+0000", 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3030000000000001e-06 2024-09-13T09:07:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006855+0000", 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090971750000000007 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006906+0000", 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1248000000000001e-05 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006950+0000", 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3763000000000001e-05 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:706 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 200704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997762+0000", 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4716587280000004, 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093043029999999999, 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 706 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997762+0000", 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997762+0000", 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997762+0000", 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997764+0000", 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5019999999999999e-06 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997771+0000", 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4869999999999999e-06 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997776+0000", 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8849999999999998e-06 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.006981+0000", 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092050229999999997 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007024+0000", 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3009000000000001e-05 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007066+0000", 2024-09-13T09:07:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2781999999999998e-05 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:707 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 206848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997780+0000", 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.47164114, 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094079739999999995, 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 707 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997780+0000", 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997780+0000", 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997779+0000", 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997781+0000", 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9089999999999998e-06 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997787+0000", 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3689999999999998e-06 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997792+0000", 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.955e-06 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007098+0000", 2024-09-13T09:07:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093065379999999996 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007144+0000", 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6276999999999998e-05 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007188+0000", 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3205999999999999e-05 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:708 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 212992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997795+0000", 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4716255250000003, 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009507415, 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 708 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997795+0000", 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997795+0000", 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997795+0000", 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997805+0000", 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.662e-06 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997812+0000", 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9750000000000001e-06 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997817+0000", 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007218+0000", 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094007610000000005 2024-09-13T09:07:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007258+0000", 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0453e-05 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007303+0000", 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4245000000000001e-05 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:709 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 219136~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997841+0000", 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4715795199999997, 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095819149999999999, 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 709 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997841+0000", 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997841+0000", 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997841+0000", 2024-09-13T09:07:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997844+0000", 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0429999999999999e-06 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997850+0000", 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.708e-06 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997855+0000", 2024-09-13T09:07:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8409999999999999e-06 2024-09-13T09:07:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007335+0000", 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094794200000000006 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007379+0000", 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4656000000000001e-05 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007423+0000", 2024-09-13T09:07:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3834000000000001e-05 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:710 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 225280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997860+0000", 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4715611089999996, 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096747900000000008, 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 710 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997860+0000", 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997860+0000", 2024-09-13T09:07:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997859+0000", 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997862+0000", 2024-09-13T09:07:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4619999999999999e-06 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997868+0000", 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9939999999999997e-06 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997872+0000", 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.968e-06 2024-09-13T09:07:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007453+0000", 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095814290000000007 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007493+0000", 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0391e-05 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007534+0000", 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0989999999999999e-05 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:711 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 231424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997876+0000", 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4715449530000004, 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098663190000000001, 2024-09-13T09:07:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 711 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997876+0000", 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997876+0000", 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997875+0000", 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997878+0000", 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2510000000000001e-06 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997883+0000", 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997888+0000", 2024-09-13T09:07:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7910000000000004e-06 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007568+0000", 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096808020000000005 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007698+0000", 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013000899999999999 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007742+0000", 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3720000000000002e-05 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:712 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997892+0000", 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4715283250000004, 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010014729, 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 712 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997892+0000", 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997892+0000", 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997892+0000", 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997894+0000", 2024-09-13T09:07:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3460000000000001e-06 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997900+0000", 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9909999999999999e-06 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997905+0000", 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5990000000000004e-06 2024-09-13T09:07:25.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007778+0000", 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098734119999999998 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007827+0000", 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8983999999999997e-05 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007907+0000", 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9887000000000003e-05 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:713 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997913+0000", 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4715079329999998, 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01025295, 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 713 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997913+0000", 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997913+0000", 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997912+0000", 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997914+0000", 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.114e-06 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997920+0000", 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5450000000000003e-06 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997932+0000", 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2146e-05 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.007957+0000", 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024358000000001 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008072+0000", 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115931 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008166+0000", 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3356000000000005e-05 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:714 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 249856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997945+0000", 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4714759810000002, 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010338592000000001, 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 714 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997945+0000", 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997945+0000", 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997944+0000", 2024-09-13T09:07:25.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997947+0000", 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3719999999999999e-06 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997951+0000", 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997954+0000", 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7480000000000002e-06 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008199+0000", 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010245687 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008240+0000", 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0729999999999998e-05 2024-09-13T09:07:25.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008283+0000", 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3424000000000003e-05 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:715 3.f 3:f7d25578:::rbd_data.1124ffa4351.0000000000000160:5 [read 256000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:20.997956+0000", 2024-09-13T09:07:25.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.4714650640000002, 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010529686, 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 715 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997956+0000", 2024-09-13T09:07:25.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997956+0000", 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997955+0000", 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997957+0000", 2024-09-13T09:07:25.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.218e-06 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997960+0000", 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.982e-06 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:20.997963+0000", 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-06 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008313+0000", 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010349975000000001 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008398+0000", 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5719999999999999e-05 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.008485+0000", 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6916999999999995e-05 2024-09-13T09:07:25.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:6319 3.15 3:ad9247e3:::rbd_data.11242ae5e73a.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 15086098~48278] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.751474+0000", 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.71794717200000002, 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013819398, 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6319 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.751474+0000", 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.751474+0000", 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.751472+0000", 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.753084+0000", 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016118009999999999 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.753109+0000", 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5486999999999998e-05 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.753119+0000", 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6010000000000001e-06 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.753219+0000", 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9702000000000002e-05 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.753476+0000", 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025754499999999997 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.754090+0000", 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061386500000000001 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.763712+0000", 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096223960000000001 2024-09-13T09:07:25.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.765033+0000", 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013208930000000001 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.765136+0000", 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010231200000000001 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.765293+0000", 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015742299999999999 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4388.0:7648 3.1d e30/22 3:bd5c174d:::rbd_data.11242ae5e73a.000000000000000c:head v 30'26, mlcod=30'24)", 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.855741+0000", 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.61367990900000002, 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010826608, 2024-09-13T09:07:25.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.855741+0000", 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.855741+0000", 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.855740+0000", 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.857213+0000", 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014732899999999999 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.857230+0000", 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6676000000000001e-05 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.857241+0000", 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1225e-05 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.857291+0000", 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9968e-05 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.857314+0000", 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2996000000000001e-05 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866512+0000", 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091984920000000008 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866567+0000", 2024-09-13T09:07:25.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5031999999999998e-05 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4388.0:7658 3.13 e30/22 3:cd10ac37:::rbd_data.11242ae5e73a.0000000000000008:head v 30'20, mlcod=30'18)", 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.871512+0000", 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.59790887100000001, 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014622660000000001, 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871512+0000", 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871512+0000", 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871510+0000", 2024-09-13T09:07:25.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.874361+0000", 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028510889999999998 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.874390+0000", 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8920000000000001e-05 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.874415+0000", 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5012999999999999e-05 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.874480+0000", 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5167999999999997e-05 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.874508+0000", 2024-09-13T09:07:25.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7668e-05 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886082+0000", 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011574398 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886135+0000", 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2485000000000003e-05 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7943 3.15 3:ad9247e3:::rbd_data.11242ae5e73a.0000000000000005:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:25.347723+0000", 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.121697588, 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0088960800000000007, 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:25.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7943 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347723+0000", 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347723+0000", 2024-09-13T09:07:25.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347722+0000", 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347727+0000", 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2080000000000002e-06 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347737+0000", 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0006e-05 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347744+0000", 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0700000000000001e-06 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.350152+0000", 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024085690000000002 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.350232+0000", 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0179999999999995e-05 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.350410+0000", 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000177708 2024-09-13T09:07:25.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.355435+0000", 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050250249999999998 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.356532+0000", 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010971869999999999 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.356581+0000", 2024-09-13T09:07:25.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9082999999999997e-05 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.356619+0000", 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7768999999999997e-05 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:25.572 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:25.574 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:25.575 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-13T09:07:25.575 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:25.568+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:25.675 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:25.672+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:25.697 INFO:teuthology.orchestra.run.smithi031.stdout:322 write 0x2ede95b thru 0x2eecde2 (0xe488 bytes) 2024-09-13T09:07:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.716 INFO:teuthology.orchestra.run.smithi031.stdout:324 trunc from 0xec26668 to 0xd130349 2024-09-13T09:07:25.746 INFO:teuthology.orchestra.run.smithi031.stdout:325 writesame 0x721bbb4 thru 0x7228046 data_size 0x4f(0xc493 bytes) 2024-09-13T09:07:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:327 write 0x5a91c01 thru 0x5a93b5f (0x1f5f bytes) 2024-09-13T09:07:25.761 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:25.762 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:25.763 INFO:teuthology.orchestra.run.smithi031.stdout:328 writesame 0x326f281 thru 0x326fd99 data_size 0x1(0xb19 bytes) 2024-09-13T09:07:25.774 INFO:teuthology.orchestra.run.smithi031.stdout:330 punch from 0xce1abea to 0xce1d2dd, (0x26f3 bytes) 2024-09-13T09:07:25.774 INFO:teuthology.orchestra.run.smithi031.stdout:331 read 0xd9f303 thru 0xda8634 (0x9332 bytes) 2024-09-13T09:07:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:332 read 0x15f0e8b thru 0x15f6c60 (0x5dd6 bytes) 2024-09-13T09:07:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:333 punch from 0xa9ab140 to 0xa9b5895, (0xa755 bytes) 2024-09-13T09:07:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:336 writesame 0x4563a33 thru 0x456f805 data_size 0x1(0xbdd3 bytes) 2024-09-13T09:07:25.777 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:25.770+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:25.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:25.816 INFO:teuthology.orchestra.run.smithi031.stdout:337 read 0x459769a thru 0x459dc53 (0x65ba bytes) 2024-09-13T09:07:25.818 INFO:teuthology.orchestra.run.smithi031.stdout:338 writesame 0x4e45295 thru 0x4e45c32 data_size 0x1(0x99e bytes) 2024-09-13T09:07:25.829 INFO:teuthology.orchestra.run.smithi031.stdout:340 punch from 0x1e6be45 to 0x1e79db7, (0xdf72 bytes) 2024-09-13T09:07:25.829 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.829 INFO:teuthology.orchestra.run.smithi031.stdout:342 read 0xa91dcc1 thru 0xa9269dc (0x8d1c bytes) 2024-09-13T09:07:25.830 INFO:teuthology.orchestra.run.smithi031.stdout:343 write 0x810cdf3 thru 0x811cc73 (0xfe81 bytes) 2024-09-13T09:07:25.838 INFO:teuthology.orchestra.run.smithi031.stdout:344 trunc from 0xd130349 to 0xd71cfc5 2024-09-13T09:07:25.847 INFO:teuthology.orchestra.run.smithi031.stdout:345 writesame 0x77090aa thru 0x7709bec data_size 0x3(0xb43 bytes) 2024-09-13T09:07:25.857 INFO:teuthology.orchestra.run.smithi031.stdout:347 writesame 0x641874 thru 0x642ec4 data_size 0x1(0x1651 bytes) 2024-09-13T09:07:25.877 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:25.874+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:25.888 INFO:teuthology.orchestra.run.smithi031.stdout:351 trunc from 0xd71cfc5 to 0x9d37a59 2024-09-13T09:07:25.931 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:25.931 INFO:teuthology.orchestra.run.smithi031.stdout:353 punch from 0x7fc489e to 0x7fc7381, (0x2ae3 bytes) 2024-09-13T09:07:25.931 INFO:teuthology.orchestra.run.smithi031.stdout:354 read 0x55fe926 thru 0x560e358 (0xfa33 bytes) 2024-09-13T09:07:25.932 INFO:teuthology.orchestra.run.smithi031.stdout:355 trunc from 0x9d37a59 to 0x9af58b6 2024-09-13T09:07:25.949 INFO:teuthology.orchestra.run.smithi031.stdout:356 write 0x2320571 thru 0x2329d42 (0x97d2 bytes) 2024-09-13T09:07:25.950 INFO:teuthology.orchestra.run.smithi031.stdout:357 write 0x7675e50 thru 0x767dfdd (0x818e bytes) 2024-09-13T09:07:25.963 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:25.963 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:25.963 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:25.966 INFO:teuthology.orchestra.run.smithi031.stdout:358 write 0xc2f8997 thru 0xc3031d8 (0xa842 bytes) 2024-09-13T09:07:25.978 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:25.972+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:25.979 INFO:teuthology.orchestra.run.smithi031.stdout:359 trunc from 0xc3031d9 to 0x5578f0e 2024-09-13T09:07:25.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:26.020 INFO:teuthology.orchestra.run.smithi031.stdout:360 punch from 0x40f986e to 0x4109081, (0xf813 bytes) 2024-09-13T09:07:26.020 INFO:teuthology.orchestra.run.smithi031.stdout:362 write 0x28615e9 thru 0x28628a7 (0x12bf bytes) 2024-09-13T09:07:26.042 INFO:teuthology.orchestra.run.smithi031.stdout:365 write 0xd10b020 thru 0xd10dcb7 (0x2c98 bytes) 2024-09-13T09:07:26.049 INFO:teuthology.orchestra.run.smithi031.stdout:366 read 0x46360a1 thru 0x4643416 (0xd376 bytes) 2024-09-13T09:07:26.073 INFO:teuthology.orchestra.run.smithi031.stdout:367 trunc from 0xd10dcb8 to 0xfd3dd4 2024-09-13T09:07:26.079 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:26.076+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:26.143 INFO:teuthology.orchestra.run.smithi031.stdout:368 read 0x395ade thru 0x39704a (0x156d bytes) 2024-09-13T09:07:26.164 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:26.164 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.468652813999999, 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840076, 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 34 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:26.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694022+0000", 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694033+0000", 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.116e-05 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694051+0000", 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694065+0000", 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3716e-05 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700273+0000", 2024-09-13T09:07:26.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062082320000000002 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700405+0000", 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131416 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700608+0000", 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020368099999999999 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705556+0000", 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049475760000000004 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705654+0000", 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8444999999999997e-05 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705741+0000", 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6416999999999996e-05 2024-09-13T09:07:26.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705863+0000", 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121876 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.468584976999999, 2024-09-13T09:07:26.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:26.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.468276963000001, 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:26.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:26.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.183 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.468189049999999, 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:26.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:26.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.197664689, 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:26.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:26.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.197576926, 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:26.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:26.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.197524872999999, 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:26.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.197339791000001, 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:26.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:26.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.197265302000002, 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:26.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:26.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.197210982000001, 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:26.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:26.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:26.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.197012897, 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.196931633999998, 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:26.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:26.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:26.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.702636919, 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:26.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:26.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.702321282, 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:26.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:26.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:26.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.702260952, 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.702203046000001, 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:26.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:26.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:26.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:26.265 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.701900449, 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:26.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:26.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:26.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7649 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.3093352819999999, 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013081786, 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7649 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.272 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853339+0000", 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853345+0000", 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853365+0000", 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9054000000000001e-05 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853374+0000", 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858418+0000", 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050442869999999997 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858525+0000", 2024-09-13T09:07:26.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010602099999999999 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858992+0000", 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046758299999999999 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.861712+0000", 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027195700000000001 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866315+0000", 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046037439999999999 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866381+0000", 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5121000000000002e-05 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866422+0000", 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1331000000000001e-05 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7657 3.10 3:08eae27d:::rbd_data.11242ae5e73a.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13762560~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.2945429879999999, 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018927840000000001, 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7657 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868131+0000", 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870832+0000", 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027013660000000002 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870853+0000", 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.107e-05 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870862+0000", 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7830000000000004e-06 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870910+0000", 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-09-13T09:07:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871083+0000", 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017277500000000001 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871386+0000", 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030378600000000001 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878951+0000", 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075642649999999997 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886982+0000", 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080316160000000001 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887025+0000", 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3288999999999998e-05 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887060+0000", 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4706999999999998e-05 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7660 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.2917998989999999, 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016257230000000001, 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7660 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870874+0000", 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870879+0000", 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:07:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870885+0000", 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870888+0000", 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876842+0000", 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059541780000000001 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876952+0000", 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109577 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.877351+0000", 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039971899999999999 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878995+0000", 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001643179 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887073+0000", 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080780109999999995 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887100+0000", 2024-09-13T09:07:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7585000000000001e-05 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887133+0000", 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2533000000000002e-05 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:26.290 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:26.291 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:26.291 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:07:26.291 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:26.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-13T09:07:26.292 INFO:teuthology.orchestra.run.smithi031.stdout:369 trunc from 0xfd3dd4 to 0xdad0288 2024-09-13T09:07:26.292 INFO:teuthology.orchestra.run.smithi031.stdout:370 writesame 0x468bd4a thru 0x4692efe data_size 0x1f(0x71b5 bytes) 2024-09-13T09:07:26.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:26.294 INFO:teuthology.orchestra.run.smithi031.stdout:371 write 0xe5c24c thru 0xe5d897 (0x164c bytes) 2024-09-13T09:07:26.294 INFO:teuthology.orchestra.run.smithi031.stdout:372 punch from 0x2180766 to 0x218b862, (0xb0fc bytes) 2024-09-13T09:07:26.294 INFO:teuthology.orchestra.run.smithi031.stdout:374 punch from 0x4495ba5 to 0x449c398, (0x67f3 bytes) 2024-09-13T09:07:26.294 INFO:teuthology.orchestra.run.smithi031.stdout:375 trunc from 0xdad0288 to 0x7aa119b 2024-09-13T09:07:26.294 INFO:teuthology.orchestra.run.smithi031.stdout:376 clone 6 order 20 su 16384 sc 9 2024-09-13T09:07:26.295 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:26.288+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:26.393 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:26.388+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:26.462 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:26.462 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:26.462 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:26.463 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:26.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:26.494 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:26.490+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:26.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:26.595 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:26.592+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:26.641 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:26.642 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:26.652 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-13T09:07:26.648+0000 7f426cbbd640 -1 mon.a@0(leader).osd e31 definitely_dead 0 2024-09-13T09:07:26.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:26.658 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:26.658 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:26.675 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:26.675 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:26.692 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:26.692 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:26.695 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:26.692+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:26.709 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:26.709 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:26.728 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:26.728 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:26.748 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:26.748 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:26.768 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:26.768 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:26.774 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:26.774 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:26.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:26.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:26.789 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:26.806 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:26.800+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:26.897 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:26.892+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:26.973 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:26.973 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:26.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:26.998 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:26.994+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:27.099 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:27.094+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:27.160 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:27.160 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:27.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:27.200 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:27.198+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:27.221 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.1. 2024-09-13T09:07:27.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:07:27.300 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:27.300 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:27.301 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:27.296+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:27.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:27.402 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:27.396+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:27.452 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:27.452 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:27.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:27.512 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:27.510+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:27.592 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:27.592 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:27.604 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:27.602+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:27.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:27.705 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:27.700+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:27.776 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:27.776 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:27.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:27.833 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:27.830+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:27.909 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:27.906+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:27.914 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:27.915 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:27.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:28.009 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:28.004+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:28.101 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:28.101 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:28.110 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:28.106+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:28.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:28.210 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:28.206+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:28.231 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:07:28.303 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:28.304 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:28.311 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:28.306+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:28.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:28.412 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:28.410+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:28.483 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:28.483 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:28.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:28.514 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:28.508+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:28.546 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone5 from 0x7aa119b (overlap 0xfd3dd4) to 0x97f1e5 2024-09-13T09:07:28.614 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:28.610+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:28.672 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:28.673 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:28.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:28.715 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:28.712+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:28.720 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-13T09:07:28.723 INFO:teuthology.orchestra.run.smithi031.stdout:378 read 0x563c537 thru 0x56483cb (0xbe95 bytes) 2024-09-13T09:07:28.724 INFO:teuthology.orchestra.run.smithi031.stdout:380 write 0x5e2db79 thru 0x5e35403 (0x788b bytes) 2024-09-13T09:07:28.748 INFO:teuthology.orchestra.run.smithi031.stdout:381 write 0x80ffdc1 thru 0x810c7fb (0xca3b bytes) 2024-09-13T09:07:28.758 INFO:teuthology.orchestra.run.smithi031.stdout:383 read 0x6f1ebe0 thru 0x6f234aa (0x48cb bytes) 2024-09-13T09:07:28.758 INFO:teuthology.orchestra.run.smithi031.stdout:384 write 0x1a11b15 thru 0x1a1d41a (0xb906 bytes) 2024-09-13T09:07:28.780 INFO:teuthology.orchestra.run.smithi031.stdout:385 trunc from 0x810c7fc to 0x48f3233 2024-09-13T09:07:28.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:28.816 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:28.814+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:28.819 INFO:teuthology.orchestra.run.smithi031.stdout:386 write 0x1fbe3d0 thru 0x1fcc0f0 (0xdd21 bytes) 2024-09-13T09:07:28.831 INFO:teuthology.orchestra.run.smithi031.stdout:387 writesame 0x40fcf9a thru 0x41037bb data_size 0x9(0x6822 bytes) 2024-09-13T09:07:28.853 INFO:teuthology.orchestra.run.smithi031.stdout:388 read 0x41a99e1 thru 0x41b5698 (0xbcb8 bytes) 2024-09-13T09:07:28.854 INFO:teuthology.orchestra.run.smithi031.stdout:389 write 0x2219f9c thru 0x2228d16 (0xed7b bytes) 2024-09-13T09:07:28.861 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:28.861 INFO:teuthology.orchestra.run.smithi031.stdout:392 read 0xb08c34 thru 0xb12159 (0x9526 bytes) 2024-09-13T09:07:28.873 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:28.873 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:28.883 INFO:teuthology.orchestra.run.smithi031.stdout:395 write 0x9c7c7cf thru 0x9c7fae5 (0x3317 bytes) 2024-09-13T09:07:28.887 INFO:teuthology.orchestra.run.smithi031.stdout:396 writesame 0x484200c thru 0x48465bd data_size 0xb(0x45b2 bytes) 2024-09-13T09:07:28.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:28.891 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:28.902 INFO:teuthology.orchestra.run.smithi031.stdout:398 writesame 0xb7c25e2 thru 0xb7ccc30 data_size 0xd(0xa64f bytes) 2024-09-13T09:07:28.902 INFO:teuthology.orchestra.run.smithi031.stdout:399 write 0xa323d8b thru 0xa330e01 (0xd077 bytes) 2024-09-13T09:07:28.911 INFO:teuthology.orchestra.run.smithi031.stdout:401 read 0x22ae4b3 thru 0x22af507 (0x1055 bytes) 2024-09-13T09:07:28.912 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:28.912 INFO:teuthology.orchestra.run.smithi031.stdout:403 read 0x814dfc4 thru 0x815b32d (0xd36a bytes) 2024-09-13T09:07:28.912 INFO:teuthology.orchestra.run.smithi031.stdout:405 write 0x85c0e16 thru 0x85cefb1 (0xe19c bytes) 2024-09-13T09:07:28.917 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:28.912+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:28.919 INFO:teuthology.orchestra.run.smithi031.stdout:407 write 0x762cbb1 thru 0x763889d (0xbced bytes) 2024-09-13T09:07:28.929 INFO:teuthology.orchestra.run.smithi031.stdout:408 write 0xa91224a thru 0xa917302 (0x50b9 bytes) 2024-09-13T09:07:28.931 INFO:teuthology.orchestra.run.smithi031.stdout:409 read 0x237b53f thru 0x2384218 (0x8cda bytes) 2024-09-13T09:07:28.932 INFO:teuthology.orchestra.run.smithi031.stdout:410 write 0x24857cd thru 0x2494f64 (0xf798 bytes) 2024-09-13T09:07:28.942 INFO:teuthology.orchestra.run.smithi031.stdout:411 write 0x27d36ef thru 0x27da686 (0x6f98 bytes) 2024-09-13T09:07:28.948 INFO:teuthology.orchestra.run.smithi031.stdout:413 punch from 0xaa38b2d to 0xaa465e0, (0xdab3 bytes) 2024-09-13T09:07:28.953 INFO:teuthology.orchestra.run.smithi031.stdout:417 trunc from 0xb7ccc31 to 0x8e01a6e 2024-09-13T09:07:28.973 INFO:teuthology.orchestra.run.smithi031.stdout:418 trunc from 0x8e01a6e to 0x3e9d09d 2024-09-13T09:07:28.994 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:28.994 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:29.019 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:29.016+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:29.020 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:29.020 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:29.037 INFO:teuthology.orchestra.run.smithi031.stdout:419 read 0x1036cc9 thru 0x103d38a (0x66c2 bytes) 2024-09-13T09:07:29.037 INFO:teuthology.orchestra.run.smithi031.stdout:420 read 0x3d36bb6 thru 0x3d389a1 (0x1dec bytes) 2024-09-13T09:07:29.037 INFO:teuthology.orchestra.run.smithi031.stdout:421 read 0x32dc986 thru 0x32e677b (0x9df6 bytes) 2024-09-13T09:07:29.037 INFO:teuthology.orchestra.run.smithi031.stdout:424 punch from 0x121c316 to 0x122662d, (0xa317 bytes) 2024-09-13T09:07:29.039 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:29.039 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:425 writesame 0x7289476 thru 0x728a536 data_size 0x1(0x10c1 bytes) 2024-09-13T09:07:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:426 punch from 0x5844b73 to 0x5846be2, (0x206f bytes) 2024-09-13T09:07:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:433 read 0x31f6674 thru 0x31f82d2 (0x1c5f bytes) 2024-09-13T09:07:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:435 read 0x358004a thru 0x3586617 (0x65ce bytes) 2024-09-13T09:07:29.050 INFO:teuthology.orchestra.run.smithi031.stdout:436 punch from 0xba818 to 0xc24bf, (0x7ca7 bytes) 2024-09-13T09:07:29.051 INFO:teuthology.orchestra.run.smithi031.stdout:437 read 0x4ddea3 thru 0x4df4a8 (0x1606 bytes) 2024-09-13T09:07:29.051 INFO:teuthology.orchestra.run.smithi031.stdout:438 read 0x4074f1 thru 0x415177 (0xdc87 bytes) 2024-09-13T09:07:29.052 INFO:teuthology.orchestra.run.smithi031.stdout:439 read 0x4e087a7 thru 0x4e13a82 (0xb2dc bytes) 2024-09-13T09:07:29.052 INFO:teuthology.orchestra.run.smithi031.stdout:440 write 0x58ccb7c thru 0x58cdc3c (0x10c1 bytes) 2024-09-13T09:07:29.056 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.058 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:29.058 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:29.071 INFO:teuthology.orchestra.run.smithi031.stdout:443 writesame 0xa39ed8c thru 0xa3a6247 data_size 0x1(0x74bc bytes) 2024-09-13T09:07:29.071 INFO:teuthology.orchestra.run.smithi031.stdout:444 read 0x847bb9c thru 0x8480d5d (0x51c2 bytes) 2024-09-13T09:07:29.071 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.071 INFO:teuthology.orchestra.run.smithi031.stdout:451 write 0x1e5c370 thru 0x1e68db5 (0xca46 bytes) 2024-09-13T09:07:29.071 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:29.071 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:29.077 INFO:teuthology.orchestra.run.smithi031.stdout:452 read 0x60f0581 thru 0x60f8c1a (0x869a bytes) 2024-09-13T09:07:29.079 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:29.080 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:29.088 INFO:teuthology.orchestra.run.smithi031.stdout:453 writesame 0xaaba841 thru 0xaabf635 data_size 0x1(0x4df5 bytes) 2024-09-13T09:07:29.088 INFO:teuthology.orchestra.run.smithi031.stdout:454 write 0x2aedd3f thru 0x2af2d7d (0x503f bytes) 2024-09-13T09:07:29.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:29.090 INFO:teuthology.orchestra.run.smithi031.stdout:459 read 0x81c4003 thru 0x81d354c (0xf54a bytes) 2024-09-13T09:07:29.090 INFO:teuthology.orchestra.run.smithi031.stdout:460 write 0x4c180e9 thru 0x4c1bf85 (0x3e9d bytes) 2024-09-13T09:07:29.096 INFO:teuthology.orchestra.run.smithi031.stdout:461 read 0x1cae6ff thru 0x1cbbeed (0xd7ef bytes) 2024-09-13T09:07:29.097 INFO:teuthology.orchestra.run.smithi031.stdout:463 writesame 0x5ca6438 thru 0x5cae095 data_size 0x1(0x7c5e bytes) 2024-09-13T09:07:29.120 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:29.116+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:29.132 INFO:teuthology.orchestra.run.smithi031.stdout:464 trunc from 0xaabf636 to 0x36e15ea 2024-09-13T09:07:29.157 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xee2349a 2024-09-13T09:07:29.185 INFO:teuthology.orchestra.run.smithi031.stdout:465 trunc from 0x36e15ea to 0xee2349a 2024-09-13T09:07:29.197 INFO:teuthology.orchestra.run.smithi031.stdout:466 read 0x256dc53 thru 0x2572768 (0x4b16 bytes) 2024-09-13T09:07:29.203 INFO:teuthology.orchestra.run.smithi031.stdout:467 trunc from 0xee2349a to 0xd2ee249 2024-09-13T09:07:29.215 INFO:teuthology.orchestra.run.smithi031.stdout:470 read 0xba11ea5 thru 0xba191f3 (0x734f bytes) 2024-09-13T09:07:29.221 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:29.218+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:29.225 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:29.226 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:29.230 INFO:teuthology.orchestra.run.smithi031.stdout:471 write 0xe7a226e thru 0xe7add01 (0xba94 bytes) 2024-09-13T09:07:29.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:29.251 INFO:teuthology.orchestra.run.smithi031.stdout:472 writesame 0x1cbbb99 thru 0x1ccacde data_size 0x59(0xf146 bytes) 2024-09-13T09:07:29.260 INFO:teuthology.orchestra.run.smithi031.stdout:474 write 0x8e7c8db thru 0x8e8a22d (0xd953 bytes) 2024-09-13T09:07:29.270 INFO:teuthology.orchestra.run.smithi031.stdout:476 write 0x8ac323a thru 0x8ad048b (0xd252 bytes) 2024-09-13T09:07:29.281 INFO:teuthology.orchestra.run.smithi031.stdout:478 read 0x2c3b64a thru 0x2c465f4 (0xafab bytes) 2024-09-13T09:07:29.283 INFO:teuthology.orchestra.run.smithi031.stdout:479 write 0xdbc542d thru 0xdbd53c4 (0xff98 bytes) 2024-09-13T09:07:29.293 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.293 INFO:teuthology.orchestra.run.smithi031.stdout:481 read 0x535f4dd thru 0x536e927 (0xf44b bytes) 2024-09-13T09:07:29.294 INFO:teuthology.orchestra.run.smithi031.stdout:482 read 0x52cdf8a thru 0x52d5048 (0x70bf bytes) 2024-09-13T09:07:29.294 INFO:teuthology.orchestra.run.smithi031.stdout:483 read 0x99e4264 thru 0x99ea2d0 (0x606d bytes) 2024-09-13T09:07:29.294 INFO:teuthology.orchestra.run.smithi031.stdout:485 write 0x75a06f7 thru 0x75ad032 (0xc93c bytes) 2024-09-13T09:07:29.303 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:29.304 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:29.304 INFO:teuthology.orchestra.run.smithi031.stdout:487 writesame 0x7c9e83 thru 0x7d0501 data_size 0x1(0x667f bytes) 2024-09-13T09:07:29.328 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:29.328 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:29.332 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:29.330+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:29.354 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:29.355 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:488 read 0x8af6994 thru 0x8affa35 (0x90a2 bytes) 2024-09-13T09:07:29.360 INFO:teuthology.orchestra.run.smithi031.stdout:489 write 0xde907c7 thru 0xde977a2 (0x6fdc bytes) 2024-09-13T09:07:29.367 INFO:teuthology.orchestra.run.smithi031.stdout:491 trunc from 0xe7add02 to 0xd504962 2024-09-13T09:07:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:492 punch from 0x8d606dd to 0x8d6395b, (0x327e bytes) 2024-09-13T09:07:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:493 read 0xb587b03 thru 0xb58ad55 (0x3253 bytes) 2024-09-13T09:07:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.385 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:499 punch from 0x1d5bd72 to 0x1d5cfd5, (0x1263 bytes) 2024-09-13T09:07:29.386 INFO:teuthology.orchestra.run.smithi031.stdout:500 write 0x7d8d859 thru 0x7d985d5 (0xad7d bytes) 2024-09-13T09:07:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:502 read 0x8dccee9 thru 0x8dd028c (0x33a4 bytes) 2024-09-13T09:07:29.394 INFO:teuthology.orchestra.run.smithi031.stdout:503 write 0x2a4c33 thru 0x2a75f6 (0x29c4 bytes) 2024-09-13T09:07:29.411 INFO:teuthology.orchestra.run.smithi031.stdout:504 writesame 0x4305043 thru 0x430b680 data_size 0x17(0x663e bytes) 2024-09-13T09:07:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:505 read 0x9604ae3 thru 0x9610448 (0xb966 bytes) 2024-09-13T09:07:29.419 INFO:teuthology.orchestra.run.smithi031.stdout:506 write 0x141b85b thru 0x141f94b (0x40f1 bytes) 2024-09-13T09:07:29.422 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:29.416+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:29.424 INFO:teuthology.orchestra.run.smithi031.stdout:507 write 0x7ae4119 thru 0x7aee4fa (0xa3e2 bytes) 2024-09-13T09:07:29.424 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:29.424 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:870 3.0 e27/22 3:04608615:::rbd_data.1124f3df56b6.0000000000000197:head v 27'4, mlcod=26'2)", 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.056781+0000", 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 8.3659495150000005, 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019374080000000001, 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056781+0000", 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056781+0000", 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056779+0000", 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056848+0000", 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9363000000000007e-05 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056891+0000", 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2806000000000003e-05 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.056911+0000", 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0293e-05 2024-09-13T09:07:29.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.057038+0000", 2024-09-13T09:07:29.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012656200000000001 2024-09-13T09:07:29.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.057105+0000", 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7377000000000002e-05 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.058627+0000", 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001521941 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.058719+0000", 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1586e-05 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:873 3.c e27/22 3:32bef489:::rbd_data.1124f3df56b6.00000000000001c8:head v 27'3, mlcod=25'1)", 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.068091+0000", 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 8.3546402759999996, 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00216622, 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068091+0000", 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068091+0000", 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068090+0000", 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068122+0000", 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2410000000000003e-05 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068147+0000", 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4426e-05 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068179+0000", 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1956999999999999e-05 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068257+0000", 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8429000000000004e-05 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.068294+0000", 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7200000000000003e-05 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.070149+0000", 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001854252 2024-09-13T09:07:29.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.070257+0000", 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108299 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:905 3.c e27/22 3:32bef489:::rbd_data.1124f3df56b6.00000000000001c8:head v 27'5, mlcod=27'3)", 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.145074+0000", 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 8.2776573469999999, 2024-09-13T09:07:29.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021751930000000002, 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145074+0000", 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145074+0000", 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145072+0000", 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145130+0000", 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8075999999999998e-05 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145179+0000", 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8943000000000002e-05 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145199+0000", 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0395999999999999e-05 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145335+0000", 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013600900000000001 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.145390+0000", 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4639999999999999e-05 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.147144+0000", 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017538860000000001 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.147249+0000", 2024-09-13T09:07:29.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105309 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:1074 3.c e28/22 3:35dadade:::rbd_data.1124f3df56b6.000000000000023a:head v 28'8, mlcod=27'6)", 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.354278+0000", 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 8.0684528340000004, 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001869735, 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354278+0000", 2024-09-13T09:07:29.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354278+0000", 2024-09-13T09:07:29.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354277+0000", 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354326+0000", 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8519000000000001e-05 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354350+0000", 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4426999999999998e-05 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354365+0000", 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4496e-05 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354492+0000", 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012715599999999999 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.354540+0000", 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7516999999999999e-05 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.356064+0000", 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015247310000000001 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.356148+0000", 2024-09-13T09:07:29.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3559000000000004e-05 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:1169 3.b 3:d76e9a83:::rbd_data.1124f3df56b6.00000000000000a2:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 181276~3044] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.443678+0000", 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 7.9790531979999999, 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032811250000000002, 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:29.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 1169 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443678+0000", 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443678+0000", 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443675+0000", 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999971 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443693+0000", 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8318999999999999e-05 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443727+0000", 2024-09-13T09:07:29.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4164e-05 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443743+0000", 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5423000000000001e-05 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.443878+0000", 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013483200000000001 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.444184+0000", 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030661099999999998 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.444549+0000", 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036525099999999998 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446190+0000", 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001640697 2024-09-13T09:07:29.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446799+0000", 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00060851500000000001 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446897+0000", 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8202000000000006e-05 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.446959+0000", 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2124000000000004e-05 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:1187 3.c e28/22 3:32a44471:::rbd_data.1124f3df56b6.000000000000012b:head v 28'11, mlcod=28'8)", 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:21.468266+0000", 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 7.9544649569999999, 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001920798, 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.450 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468266+0000", 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468266+0000", 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468264+0000", 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468281+0000", 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7073000000000001e-05 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468318+0000", 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6396000000000002e-05 2024-09-13T09:07:29.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468336+0000", 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8099999999999999e-05 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468625+0000", 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028886500000000002 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.468702+0000", 2024-09-13T09:07:29.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6934999999999994e-05 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.470108+0000", 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014062429999999999 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:21.470187+0000", 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9012999999999997e-05 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:6371 3.b 3:d31ec9c7:::rbd_data.11242aa72797.0000000000000028:7 [sparse-read 917504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:24.769469+0000", 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.6532615069999999, 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001900824, 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 6371 2024-09-13T09:07:29.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.769469+0000", 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.769469+0000", 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.769469+0000", 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.769473+0000", 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7850000000000002e-06 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.769485+0000", 2024-09-13T09:07:29.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2724999999999999e-05 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.769493+0000", 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9899999999999997e-06 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.771164+0000", 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016705590000000001 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.771261+0000", 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7113000000000003e-05 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.771370+0000", 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109184 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:7641 3.0 e30/22 3:06127288:::rbd_data.11242ae5e73a.0000000000000002:head v 30'12, mlcod=30'10)", 2024-09-13T09:07:29.459 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:24.851523+0000", 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.5712079780000003, 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0073102210000000004, 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.851523+0000", 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.851523+0000", 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.851522+0000", 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.853357+0000", 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018350669999999999 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.853381+0000", 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4845e-05 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.853393+0000", 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.13e-05 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.853489+0000", 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5938000000000002e-05 2024-09-13T09:07:29.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.853517+0000", 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8595e-05 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.858776+0000", 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0052586689999999997 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.858833+0000", 2024-09-13T09:07:29.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7237000000000001e-05 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:7688 3.0 e30/22 3:06127288:::rbd_data.11242ae5e73a.0000000000000002:head v 30'13, mlcod=30'11)", 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:24.983708+0000", 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.4390233820000002, 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0049467349999999998, 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.983708+0000", 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.983708+0000", 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.983705+0000", 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:29.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.983723+0000", 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8110000000000001e-05 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.983767+0000", 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3853000000000002e-05 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.983788+0000", 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0883e-05 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.983893+0000", 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105122 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.983950+0000", 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6573000000000002e-05 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.988551+0000", 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0046009049999999998 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.988654+0000", 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010364199999999999 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:7689 3.c e30/22 3:3262ca46:::rbd_data.11242ae5e73a.0000000000000003:head v 30'16, mlcod=30'14)", 2024-09-13T09:07:29.468 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:24.984541+0000", 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.4381903510000003, 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002972326, 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.984541+0000", 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.984541+0000", 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.984539+0000", 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.984557+0000", 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8246000000000001e-05 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.984589+0000", 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2465999999999997e-05 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.984609+0000", 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.986e-05 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.984797+0000", 2024-09-13T09:07:29.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018801199999999999 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.984849+0000", 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2120000000000002e-05 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.987385+0000", 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025359430000000001 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:24.987513+0000", 2024-09-13T09:07:29.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000127599 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_update_log_missing(3.0 epoch 30/22 rep_tid 94 entries 30'14 (0'0) error 3:61272880:::rbd_data.11242ae5e73a.0000000000000002:head by client.4388.0:7724 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 30'12)", 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:25.043178+0000", 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.3795531639999998, 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001905215, 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:07:29.473 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043178+0000", 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043178+0000", 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043176+0000", 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043198+0000", 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2637e-05 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043263+0000", 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5235999999999997e-05 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043300+0000", 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6106000000000003e-05 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043575+0000", 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027511499999999998 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.045083+0000", 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001508328 2024-09-13T09:07:29.476 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_update_log_missing(3.c epoch 30/22 rep_tid 44 entries 30'17 (0'0) error 3:3262ca46:::rbd_data.11242ae5e73a.0000000000000003:head by client.4388.0:7725 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 30'15)", 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:25.043609+0000", 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.3791219379999999, 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022047960000000002, 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043609+0000", 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043609+0000", 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043607+0000", 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043629+0000", 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1532999999999999e-05 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043669+0000", 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0450000000000001e-05 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043704+0000", 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5113000000000001e-05 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.043961+0000", 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025702500000000003 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.045814+0000", 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018525309999999999 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.480 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_update_log_missing(3.c epoch 30/22 rep_tid 48 entries 30'18 (0'0) error 3:3262ca46:::rbd_data.11242ae5e73a.0000000000000003:head by client.4388.0:7771 0.000000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 30'16)", 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:25.105174+0000", 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.3175572449999997, 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002440911, 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.105174+0000", 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.105174+0000", 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.105172+0000", 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.105196+0000", 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3759000000000001e-05 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.105240+0000", 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4889999999999999e-05 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.105280+0000", 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9740000000000002e-05 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.105619+0000", 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033923299999999998 2024-09-13T09:07:29.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.107615+0000", 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001995209 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:7941 3.c e31/22 3:3262ca46:::rbd_data.11242ae5e73a.0000000000000003:head v 31'20, mlcod=30'17)", 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:25.347604+0000", 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.0751272710000004, 2024-09-13T09:07:29.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0063084120000000002, 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.347604+0000", 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.347604+0000", 2024-09-13T09:07:29.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.347603+0000", 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.347628+0000", 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5443999999999999e-05 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.347663+0000", 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4496000000000003e-05 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.347685+0000", 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2133e-05 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.347876+0000", 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000190612 2024-09-13T09:07:29.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.347941+0000", 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5436000000000002e-05 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.353827+0000", 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058863589999999999 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.353912+0000", 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4821999999999995e-05 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:7940 3.0 e31/22 3:06127288:::rbd_data.11242ae5e73a.0000000000000002:head v 31'18, mlcod=30'15)", 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:25.348259+0000", 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.0744721559999997, 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0065633239999999997, 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.348259+0000", 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.348259+0000", 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.348258+0000", 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.348270+0000", 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1674e-05 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.348314+0000", 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4039999999999998e-05 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.348333+0000", 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9409e-05 2024-09-13T09:07:29.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.348545+0000", 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021209199999999999 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.348595+0000", 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9475999999999999e-05 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.354746+0000", 2024-09-13T09:07:29.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0061511459999999997 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.354822+0000", 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6377000000000004e-05 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:9182 3.6 e31/22 3:647f5c28:::rbd_data.1124fee72b32.000000000000000b:head v 31'9, mlcod=29'6)", 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:25.711514+0000", 2024-09-13T09:07:29.493 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.7112172220000001, 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002098291, 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.711514+0000", 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.711514+0000", 2024-09-13T09:07:29.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.711513+0000", 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.711602+0000", 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9012999999999996e-05 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.711631+0000", 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8595999999999999e-05 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.711646+0000", 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5003e-05 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.711765+0000", 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119583 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.711811+0000", 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5333000000000001e-05 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.713516+0000", 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001705367 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.713612+0000", 2024-09-13T09:07:29.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6106000000000004e-05 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:9307 3.0 e31/22 3:054fa7a2:::rbd_data.1124fee72b32.0000000000000018:head v 31'19, mlcod=30'16)", 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:25.731587+0000", 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.6911436169999998, 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021701049999999999, 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.731587+0000", 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.731587+0000", 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.731585+0000", 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.731600+0000", 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4905999999999999e-05 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.731629+0000", 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8974000000000001e-05 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.731648+0000", 2024-09-13T09:07:29.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9130000000000001e-05 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.731773+0000", 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012465499999999999 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.731832+0000", 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9222999999999999e-05 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.733654+0000", 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018213439999999999 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:25.733757+0000", 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010395600000000001 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:10025 3.0 e31/22 3:054fa7a2:::rbd_data.1124fee72b32.0000000000000018:head v 31'24, mlcod=31'22)", 2024-09-13T09:07:29.502 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:26.047081+0000", 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.3756496990000002, 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002070934, 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.047081+0000", 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.047081+0000", 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.047079+0000", 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999971 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.047143+0000", 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4216999999999999e-05 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.047185+0000", 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2475999999999998e-05 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.047207+0000", 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1676e-05 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.047324+0000", 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011710299999999999 2024-09-13T09:07:29.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.047387+0000", 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2910000000000006e-05 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.049056+0000", 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001668973 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.049152+0000", 2024-09-13T09:07:29.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6238999999999998e-05 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:10121 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:26.112610+0000", 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.31012065, 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021755238, 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:29.507 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 10121 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.112610+0000", 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.112610+0000", 2024-09-13T09:07:29.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.112607+0000", 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999967 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.119750+0000", 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0071430340000000004 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.119800+0000", 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0040000000000002e-05 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.119825+0000", 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5486999999999998e-05 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.119977+0000", 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015178199999999999 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.120408+0000", 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043104599999999998 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.121415+0000", 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010064690000000001 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.132812+0000", 2024-09-13T09:07:29.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011396929 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.134188+0000", 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0013765959999999999 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.134300+0000", 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011231299999999999 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.134366+0000", 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5086000000000007e-05 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.4388.0:10121 3.b e31/22 ondisk, result = 0)", 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:26.130883+0000", 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.2918483589999998, 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019358520000000001, 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:29.513 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.130883+0000", 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.130883+0000", 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.130880+0000", 2024-09-13T09:07:29.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.130893+0000", 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2679e-05 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.130926+0000", 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3099999999999998e-05 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.130942+0000", 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6223e-05 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.132787+0000", 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018440100000000001 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.132801+0000", 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4722999999999999e-05 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:29.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:26.132818+0000", 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7246000000000001e-05 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:29.517 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:29.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:29.520 INFO:teuthology.orchestra.run.smithi031.stdout:508 read 0x33d0325 thru 0x33dcfab (0xcc87 bytes) 2024-09-13T09:07:29.520 INFO:teuthology.orchestra.run.smithi031.stdout:509 write 0xb1d4285 thru 0xb1e262f (0xe3ab bytes) 2024-09-13T09:07:29.520 INFO:teuthology.orchestra.run.smithi031.stdout:510 writesame 0xded9839 thru 0xdedec9b data_size 0x13(0x5463 bytes) 2024-09-13T09:07:29.520 INFO:teuthology.orchestra.run.smithi031.stdout:511 writesame 0xc2d006f thru 0xc2dd425 data_size 0x1(0xd3b7 bytes) 2024-09-13T09:07:29.520 INFO:teuthology.orchestra.run.smithi031.stdout:513 write 0xc525c28 thru 0xc53473c (0xeb15 bytes) 2024-09-13T09:07:29.520 INFO:teuthology.orchestra.run.smithi031.stdout:514 read 0x8892c08 thru 0x88974b6 (0x48af bytes) 2024-09-13T09:07:29.520 INFO:teuthology.orchestra.run.smithi031.stdout:515 writesame 0xced7e9f thru 0xcede08e data_size 0x8(0x61f0 bytes) 2024-09-13T09:07:29.521 INFO:teuthology.orchestra.run.smithi031.stdout:517 write 0xb442686 thru 0xb44e3a0 (0xbd1b bytes) 2024-09-13T09:07:29.521 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.540 INFO:teuthology.orchestra.run.smithi031.stdout:519 trunc from 0xdedec9c to 0x5b801db 2024-09-13T09:07:29.561 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:29.558+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:29.580 INFO:teuthology.orchestra.run.smithi031.stdout:522 read 0x93c58f thru 0x94bf70 (0xf9e2 bytes) 2024-09-13T09:07:29.590 INFO:teuthology.orchestra.run.smithi031.stdout:523 trunc from 0x5b801db to 0xa3e8ba0 2024-09-13T09:07:29.602 INFO:teuthology.orchestra.run.smithi031.stdout:525 write 0x65ef421 thru 0x65ef53d (0x11d bytes) 2024-09-13T09:07:29.608 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:29.608 INFO:teuthology.orchestra.run.smithi031.stdout:528 writesame 0x5ca6cff thru 0x5cb09a4 data_size 0x1(0x9ca6 bytes) 2024-09-13T09:07:29.624 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:29.620+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:29.650 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:29.650 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:29.650 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:29.650 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:29.659 INFO:teuthology.orchestra.run.smithi031.stdout:529 read 0x70189bb thru 0x702751d (0xeb63 bytes) 2024-09-13T09:07:29.659 INFO:teuthology.orchestra.run.smithi031.stdout:530 writesame 0x6f4a800 thru 0x6f4c781 data_size 0x25(0x1f82 bytes) 2024-09-13T09:07:29.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:29.668 INFO:teuthology.orchestra.run.smithi031.stdout:533 read 0x2b5cb33 thru 0x2b67bf4 (0xb0c2 bytes) 2024-09-13T09:07:29.670 INFO:teuthology.orchestra.run.smithi031.stdout:534 read 0x44d85f4 thru 0x44e192e (0x933b bytes) 2024-09-13T09:07:29.671 INFO:teuthology.orchestra.run.smithi031.stdout:536 punch from 0xaa9caa to 0xab1d94, (0x80ea bytes) 2024-09-13T09:07:29.671 INFO:teuthology.orchestra.run.smithi031.stdout:538 read 0x1135963 thru 0x11376b5 (0x1d53 bytes) 2024-09-13T09:07:29.671 INFO:teuthology.orchestra.run.smithi031.stdout:540 read 0x958c985 thru 0x959404d (0x76c9 bytes) 2024-09-13T09:07:29.671 INFO:teuthology.orchestra.run.smithi031.stdout:541 clone 7 order 24 su 65536 sc 3 2024-09-13T09:07:29.725 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:29.722+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:29.792 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:29.792 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:29.792 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:29.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:29.855 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:29.852+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:29.927 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:29.924+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:29.982 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:29.982 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11540 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1343488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314758+0000", 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66522982600000002, 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027958021999999999, 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:29.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11540 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314758+0000", 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314758+0000", 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:29.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314757+0000", 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314760+0000", 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7690000000000001e-06 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314766+0000", 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1689999999999997e-06 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314771+0000", 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7899999999999999e-06 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.342223+0000", 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027452014 2024-09-13T09:07:29.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.342342+0000", 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118489 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.342716+0000", 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037415000000000001 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:29.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11541 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314776+0000", 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66521206099999997, 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028398428, 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11541 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314776+0000", 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314776+0000", 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:29.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314775+0000", 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314778+0000", 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6029999999999999e-06 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314783+0000", 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314789+0000", 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1619999999999999e-06 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.342749+0000", 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027960137999999999 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.342840+0000", 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1609999999999999e-05 2024-09-13T09:07:29.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343174+0000", 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033374499999999998 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11542 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1638400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314798+0000", 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66518930499999995, 2024-09-13T09:07:29.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028618411, 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11542 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314798+0000", 2024-09-13T09:07:29.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314798+0000", 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314798+0000", 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:29.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314801+0000", 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2490000000000001e-06 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314808+0000", 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8140000000000003e-06 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314813+0000", 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2399999999999998e-06 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343221+0000", 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028407452 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343306+0000", 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5767999999999996e-05 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343417+0000", 2024-09-13T09:07:29.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110357 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11543 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1785856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314817+0000", 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66517045900000005, 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028867417999999999, 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:29.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11543 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314817+0000", 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314817+0000", 2024-09-13T09:07:29.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314817+0000", 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314819+0000", 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:29.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314825+0000", 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6350000000000003e-06 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314829+0000", 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343480+0000", 2024-09-13T09:07:30.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028650268999999999 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343569+0000", 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9732000000000001e-05 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343685+0000", 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115275 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11544 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314834+0000", 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66515405299999997, 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029327243999999999, 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11544 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314834+0000", 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314834+0000", 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314833+0000", 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314836+0000", 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314842+0000", 2024-09-13T09:07:30.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9649999999999996e-06 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314846+0000", 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8590000000000004e-06 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343727+0000", 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028880985000000001 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343815+0000", 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7590999999999995e-05 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344161+0000", 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034583500000000001 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11545 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314851+0000", 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66513690999999997, 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029748916, 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11545 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314851+0000", 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314851+0000", 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314850+0000", 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314853+0000", 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314859+0000", 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3640000000000004e-06 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314864+0000", 2024-09-13T09:07:30.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6550000000000003e-06 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344193+0000", 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029328913000000002 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344280+0000", 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7435000000000006e-05 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344600+0000", 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031923699999999999 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11546 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2228224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314868+0000", 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66511950099999995, 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030015035999999998, 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11546 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314868+0000", 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314868+0000", 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314868+0000", 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314870+0000", 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5289999999999998e-06 2024-09-13T09:07:30.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314878+0000", 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314882+0000", 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344650+0000", 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029767709999999999 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344764+0000", 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011393 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344883+0000", 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011937100000000001 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11547 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314887+0000", 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66510076200000001, 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030498834999999998, 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11547 2024-09-13T09:07:30.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314887+0000", 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314887+0000", 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314887+0000", 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314889+0000", 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9750000000000001e-06 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314892+0000", 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8310000000000003e-06 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314895+0000", 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.914e-06 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344921+0000", 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030025738 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345009+0000", 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8160999999999998e-05 2024-09-13T09:07:30.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345386+0000", 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037646899999999999 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11548 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314898+0000", 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66508996399999998, 2024-09-13T09:07:30.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030745146000000001, 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11548 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314898+0000", 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314898+0000", 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314898+0000", 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314899+0000", 2024-09-13T09:07:30.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5209999999999999e-06 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314902+0000", 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1109999999999999e-06 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314906+0000", 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2780000000000004e-06 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345434+0000", 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030527110999999999 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345518+0000", 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4875000000000003e-05 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345643+0000", 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012445699999999999 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11549 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314909+0000", 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66507896700000002, 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031201804999999999, 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11549 2024-09-13T09:07:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314909+0000", 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314909+0000", 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314908+0000", 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314910+0000", 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.46e-06 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314913+0000", 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:07:30.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314916+0000", 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345679+0000", 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030763092999999998 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345767+0000", 2024-09-13T09:07:30.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8197999999999998e-05 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346111+0000", 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034368300000000001 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11550 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314918+0000", 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66506980500000001, 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031725077999999997, 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11550 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314918+0000", 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314918+0000", 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314918+0000", 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314919+0000", 2024-09-13T09:07:30.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.367e-06 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314922+0000", 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0259999999999999e-06 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314925+0000", 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8590000000000001e-06 2024-09-13T09:07:30.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346204+0000", 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031279080000000001 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346298+0000", 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3653999999999995e-05 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346643+0000", 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034530100000000001 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11551 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.037 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314927+0000", 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66506064499999995, 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031961214000000002, 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11551 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314927+0000", 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314927+0000", 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314927+0000", 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314928+0000", 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4050000000000001e-06 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314931+0000", 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314937+0000", 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0340000000000002e-06 2024-09-13T09:07:30.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346675+0000", 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031737994999999998 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346779+0000", 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010354800000000001 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346888+0000", 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109472 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11552 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314940+0000", 2024-09-13T09:07:30.043 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66504818200000004, 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032564320000000001, 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11552 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314940+0000", 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314940+0000", 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314939+0000", 2024-09-13T09:07:30.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314941+0000", 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4440000000000001e-06 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314944+0000", 2024-09-13T09:07:30.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314946+0000", 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6029999999999999e-06 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346940+0000", 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031993332999999999 2024-09-13T09:07:30.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.347050+0000", 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010993999999999999 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.347504+0000", 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000454308 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11553 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314948+0000", 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66503926999999996, 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033136483000000001, 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11553 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314948+0000", 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314948+0000", 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314948+0000", 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314950+0000", 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:07:30.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314952+0000", 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8270000000000002e-06 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314955+0000", 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8100000000000002e-06 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.347564+0000", 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032608919 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.347722+0000", 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000157992 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348085+0000", 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036282000000000003 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11554 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3407872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314957+0000", 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66503036699999996, 2024-09-13T09:07:30.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033748444000000002, 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11554 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314957+0000", 2024-09-13T09:07:30.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314957+0000", 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314957+0000", 2024-09-13T09:07:30.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314976+0000", 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8783999999999998e-05 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314981+0000", 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:07:30.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314984+0000", 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.247e-06 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348205+0000", 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03322141 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348295+0000", 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9178999999999994e-05 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348706+0000", 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041115299999999999 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11555 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314987+0000", 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66500058200000001, 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034189124000000001, 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11555 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314987+0000", 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314987+0000", 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314987+0000", 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314988+0000", 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5349999999999999e-06 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314991+0000", 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314994+0000", 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9009999999999998e-06 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348778+0000", 2024-09-13T09:07:30.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033783852000000003 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348846+0000", 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7866000000000004e-05 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.349176+0000", 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033037000000000002 2024-09-13T09:07:30.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11556 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3702784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314996+0000", 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66499128900000004, 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035061971999999997, 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11556 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314996+0000", 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314996+0000", 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314996+0000", 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314998+0000", 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5149999999999999e-06 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315001+0000", 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3510000000000002e-06 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315005+0000", 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9670000000000003e-06 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.349229+0000", 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034223954000000001 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.349322+0000", 2024-09-13T09:07:30.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3006999999999999e-05 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350058+0000", 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073638299999999996 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11557 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3850240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.315009+0000", 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66497873200000002, 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035481024, 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11557 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315009+0000", 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315009+0000", 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315009+0000", 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315021+0000", 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315028+0000", 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0600000000000002e-06 2024-09-13T09:07:30.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315033+0000", 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350093+0000", 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035059923999999999 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350202+0000", 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010902499999999999 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350490+0000", 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000287659 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11558 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3997696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.315039+0000", 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66494897200000003, 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037167545000000003, 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11558 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315039+0000", 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315039+0000", 2024-09-13T09:07:30.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315038+0000", 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315041+0000", 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.779e-06 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315047+0000", 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.28e-06 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315052+0000", 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8810000000000004e-06 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350529+0000", 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035476925999999999 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350596+0000", 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6698000000000004e-05 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.352206+0000", 2024-09-13T09:07:30.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001610453 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11559 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.315057+0000", 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.66493076600000001, 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037313261, 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:30.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11559 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315057+0000", 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315057+0000", 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315057+0000", 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315059+0000", 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.458e-06 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315065+0000", 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8159999999999999e-06 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315077+0000", 2024-09-13T09:07:30.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.256e-05 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.352236+0000", 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037158785999999999 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.352300+0000", 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3615999999999995e-05 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.352370+0000", 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0427999999999997e-05 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.086 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:30.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:30.088 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:30.084+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:30.189 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:30.184+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:30.265 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:30.265 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:30.265 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:30.265 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:30.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:07:30.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:30.323 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:30.320+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:30.390 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:30.384+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:30.431 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone6 from 0xa3e8ba0 (overlap 0x36e15ea) to 0x15f3bf7 2024-09-13T09:07:30.465 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:30.465 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:30.465 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:30.488 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:30.535 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:30.532+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:30.592 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:30.590+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:30.665 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.96903812, 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840076, 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 34 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694022+0000", 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694033+0000", 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.116e-05 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694051+0000", 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694065+0000", 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3716e-05 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700273+0000", 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062082320000000002 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700405+0000", 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131416 2024-09-13T09:07:30.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700608+0000", 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020368099999999999 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705556+0000", 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049475760000000004 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705654+0000", 2024-09-13T09:07:30.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8444999999999997e-05 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705741+0000", 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6416999999999996e-05 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705863+0000", 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121876 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.968970282999997, 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:30.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:30.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.968662268999999, 2024-09-13T09:07:30.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:30.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:30.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:30.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.968574355999998, 2024-09-13T09:07:30.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:30.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:30.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.698049994999998, 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:30.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:30.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.697962232000002, 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:30.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:30.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.697910178999997, 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:30.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:30.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:30.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.697725097000003, 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:30.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:30.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.697650608000004, 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:30.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:30.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.697596288, 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:30.723 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:30.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.697398202999999, 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:30.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:30.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:30.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.69731694, 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:30.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:30.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.203022224999998, 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:30.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:30.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.202706587999998, 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:30.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:30.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:30.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.202646258000001, 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:30.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.202588351999999, 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:30.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:30.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:30.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.202285754999998, 2024-09-13T09:07:30.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:30.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:30.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7649 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:30.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.8097205880000002, 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013081786, 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7649 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853339+0000", 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:30.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853345+0000", 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853365+0000", 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9054000000000001e-05 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853374+0000", 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858418+0000", 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050442869999999997 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858525+0000", 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010602099999999999 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858992+0000", 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046758299999999999 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.861712+0000", 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027195700000000001 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866315+0000", 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046037439999999999 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866381+0000", 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5121000000000002e-05 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866422+0000", 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1331000000000001e-05 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7657 3.10 3:08eae27d:::rbd_data.11242ae5e73a.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13762560~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.794928294, 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018927840000000001, 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7657 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868131+0000", 2024-09-13T09:07:30.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870832+0000", 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027013660000000002 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870853+0000", 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.107e-05 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870862+0000", 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7830000000000004e-06 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870910+0000", 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871083+0000", 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017277500000000001 2024-09-13T09:07:30.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871386+0000", 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030378600000000001 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878951+0000", 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075642649999999997 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886982+0000", 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080316160000000001 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887025+0000", 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3288999999999998e-05 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887060+0000", 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4706999999999998e-05 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7660 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.792185205, 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016257230000000001, 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:30.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7660 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870874+0000", 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870879+0000", 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870885+0000", 2024-09-13T09:07:30.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870888+0000", 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876842+0000", 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059541780000000001 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876952+0000", 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109577 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.877351+0000", 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039971899999999999 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878995+0000", 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001643179 2024-09-13T09:07:30.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887073+0000", 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080780109999999995 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887100+0000", 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7585000000000001e-05 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887133+0000", 2024-09-13T09:07:30.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2533000000000002e-05 2024-09-13T09:07:30.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:30.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:30.792 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:30.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:30.794 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-13T09:07:30.794 INFO:teuthology.orchestra.run.smithi031.stdout:542 writesame 0x52950cc thru 0x52a2acf data_size 0x6(0xda04 bytes) 2024-09-13T09:07:30.794 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:30.788+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:30.817 INFO:teuthology.orchestra.run.smithi031.stdout:543 read 0x85cd190 thru 0x85d3341 (0x61b2 bytes) 2024-09-13T09:07:30.817 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:30.839 INFO:teuthology.orchestra.run.smithi031.stdout:547 write 0xe3b7c7a thru 0xe3b80de (0x465 bytes) 2024-09-13T09:07:30.844 INFO:teuthology.orchestra.run.smithi031.stdout:549 read 0x953b8d6 thru 0x953c39b (0xac6 bytes) 2024-09-13T09:07:30.845 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:30.845 INFO:teuthology.orchestra.run.smithi031.stdout:551 punch from 0x3dc2f3c to 0x3dc5b68, (0x2c2c bytes) 2024-09-13T09:07:30.845 INFO:teuthology.orchestra.run.smithi031.stdout:552 write 0x6d4d138 thru 0x6d54025 (0x6eee bytes) 2024-09-13T09:07:30.875 INFO:teuthology.orchestra.run.smithi031.stdout:553 trunc from 0xe3b80df to 0xa1217b0 2024-09-13T09:07:30.895 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:30.890+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:30.906 INFO:teuthology.orchestra.run.smithi031.stdout:554 read 0x4c4f9fb thru 0x4c5f725 (0xfd2b bytes) 2024-09-13T09:07:30.907 INFO:teuthology.orchestra.run.smithi031.stdout:555 write 0x3ca350f thru 0x3cae6f8 (0xb1ea bytes) 2024-09-13T09:07:30.923 INFO:teuthology.orchestra.run.smithi031.stdout:556 write 0xae49c8f thru 0xae4acab (0x101d bytes) 2024-09-13T09:07:30.939 INFO:teuthology.orchestra.run.smithi031.stdout:557 writesame 0xb468c9d thru 0xb474de9 data_size 0x5(0xc14d bytes) 2024-09-13T09:07:30.941 INFO:teuthology.orchestra.run.smithi031.stdout:559 write 0xab1ee06 thru 0xab2056f (0x176a bytes) 2024-09-13T09:07:30.946 INFO:teuthology.orchestra.run.smithi031.stdout:560 trunc from 0xb474dea to 0xa8d7f2c 2024-09-13T09:07:30.962 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:30.962 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:30.962 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:30.962 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:30.966 INFO:teuthology.orchestra.run.smithi031.stdout:562 trunc from 0xa8d7f2c to 0xd6f3760 2024-09-13T09:07:30.977 INFO:teuthology.orchestra.run.smithi031.stdout:563 read 0x2b74fb9 thru 0x2b8048d (0xb4d5 bytes) 2024-09-13T09:07:30.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:30.979 INFO:teuthology.orchestra.run.smithi031.stdout:564 write 0xd43ae54 thru 0xd43e177 (0x3324 bytes) 2024-09-13T09:07:30.983 INFO:teuthology.orchestra.run.smithi031.stdout:565 write 0xc1697d9 thru 0xc16d0cd (0x38f5 bytes) 2024-09-13T09:07:30.989 INFO:teuthology.orchestra.run.smithi031.stdout:566 read 0x7efffb4 thru 0x7f0d82a (0xd877 bytes) 2024-09-13T09:07:30.990 INFO:teuthology.orchestra.run.smithi031.stdout:567 write 0x387fe52 thru 0x388bba5 (0xbd54 bytes) 2024-09-13T09:07:30.995 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:30.990+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:31.037 INFO:teuthology.orchestra.run.smithi031.stdout:570 trunc from 0xd6f3760 to 0xc4c901d 2024-09-13T09:07:31.051 INFO:teuthology.orchestra.run.smithi031.stdout:571 read 0x9600768 thru 0x960640a (0x5ca3 bytes) 2024-09-13T09:07:31.052 INFO:teuthology.orchestra.run.smithi031.stdout:574 writesame 0x5826ba6 thru 0x583601a data_size 0x1(0xf475 bytes) 2024-09-13T09:07:31.097 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:31.092+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:31.129 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:31.129 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:31.129 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:31.132 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:31.132 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:31.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:31.153 INFO:teuthology.orchestra.run.smithi031.stdout:578 trunc from 0xc4c901d to 0x2d127e9 2024-09-13T09:07:31.198 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:31.196+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:31.257 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:07:31.299 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:31.294+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:31.337 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:942 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.194730+0000", 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.141147802000001, 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059192239999999998, 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 942 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.338 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194730+0000", 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194730+0000", 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194727+0000", 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194746+0000", 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8870000000000001e-05 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194796+0000", 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0046000000000002e-05 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194818+0000", 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.194e-05 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194887+0000", 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9105999999999999e-05 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195006+0000", 2024-09-13T09:07:31.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011920400000000001 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195500+0000", 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000494036 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.197706+0000", 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022059359999999999 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.200417+0000", 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027112410000000001 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.200549+0000", 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013169599999999999 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.200650+0000", 2024-09-13T09:07:31.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010048900000000001 2024-09-13T09:07:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:943 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.194887+0000", 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.140990877, 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0063994940000000004, 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 943 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194887+0000", 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194887+0000", 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194886+0000", 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:31.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194898+0000", 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2763e-05 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194924+0000", 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5749e-05 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194940+0000", 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6093999999999999e-05 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.196043+0000", 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011028539999999999 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.196145+0000", 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010233799999999999 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.196392+0000", 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024624899999999998 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.200099+0000", 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037073700000000002 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.201055+0000", 2024-09-13T09:07:31.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00095612799999999999 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.201183+0000", 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012815299999999999 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.201287+0000", 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103492 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:944 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.194974+0000", 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.140904551, 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066597289999999996, 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 944 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.351 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194974+0000", 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194974+0000", 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194972+0000", 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.194982+0000", 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8030000000000008e-06 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195028+0000", 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6207e-05 2024-09-13T09:07:31.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195048+0000", 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9522e-05 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.196837+0000", 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00178922 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.196939+0000", 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101772 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.197350+0000", 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041085500000000001 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.201334+0000", 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039838249999999999 2024-09-13T09:07:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.201418+0000", 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4642999999999996e-05 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.201538+0000", 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119589 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.201633+0000", 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5595999999999997e-05 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:945 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.195097+0000", 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.140780865, 2024-09-13T09:07:31.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0085765830000000005, 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 945 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195097+0000", 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195097+0000", 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195096+0000", 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195107+0000", 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1182999999999999e-05 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195130+0000", 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2813e-05 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.195145+0000", 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4932999999999999e-05 2024-09-13T09:07:31.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.197781+0000", 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026365099999999999 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.198018+0000", 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000236868 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.199139+0000", 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011208710000000001 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.201664+0000", 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002524727 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.203481+0000", 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018171330000000001 2024-09-13T09:07:31.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.203534+0000", 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3350000000000003e-05 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.203674+0000", 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013966799999999999 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:973 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.234081+0000", 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.101797632, 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065667310000000001, 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 973 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.364 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234081+0000", 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234081+0000", 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234080+0000", 2024-09-13T09:07:31.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234084+0000", 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2420000000000002e-06 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234092+0000", 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1890000000000001e-06 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234110+0000", 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8668000000000002e-05 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.236231+0000", 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002120481 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.236331+0000", 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100389 2024-09-13T09:07:31.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.236795+0000", 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046382299999999998 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240486+0000", 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003690872 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240560+0000", 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240610+0000", 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0260000000000003e-05 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240647+0000", 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7215000000000002e-05 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:974 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.234117+0000", 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.10176139, 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066719149999999996, 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 974 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234117+0000", 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234117+0000", 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234116+0000", 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234122+0000", 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6570000000000002e-06 2024-09-13T09:07:31.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234130+0000", 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2889999999999998e-06 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234136+0000", 2024-09-13T09:07:31.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3970000000000002e-06 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.237279+0000", 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031434480000000001 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.237378+0000", 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8616999999999994e-05 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.237832+0000", 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00045428100000000001 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240508+0000", 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026762439999999999 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240698+0000", 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018973199999999999 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240750+0000", 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2012000000000003e-05 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240789+0000", 2024-09-13T09:07:31.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8745999999999998e-05 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:975 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.234141+0000", 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.101737239, 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080235519999999998, 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 975 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234141+0000", 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234141+0000", 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234141+0000", 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234146+0000", 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2680000000000002e-06 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234153+0000", 2024-09-13T09:07:31.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1969999999999999e-06 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234159+0000", 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5230000000000003e-06 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.238635+0000", 2024-09-13T09:07:31.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044762459999999997 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.238718+0000", 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2653999999999998e-05 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.239083+0000", 2024-09-13T09:07:31.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036579400000000001 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240517+0000", 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014335470000000001 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.242066+0000", 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001549187 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.242124+0000", 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7833e-05 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.242165+0000", 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0683000000000003e-05 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:976 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.234167+0000", 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.101711278, 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.008744155, 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 976 2024-09-13T09:07:31.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234167+0000", 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234167+0000", 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234167+0000", 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234172+0000", 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.938e-06 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234180+0000", 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6580000000000008e-06 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.234186+0000", 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3010000000000003e-06 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.239524+0000", 2024-09-13T09:07:31.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053376580000000003 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.239631+0000", 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107363 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240092+0000", 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046012800000000002 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.240968+0000", 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00087658099999999997 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.242768+0000", 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018000169999999999 2024-09-13T09:07:31.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.242868+0000", 2024-09-13T09:07:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9363000000000004e-05 2024-09-13T09:07:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.242911+0000", 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3532000000000003e-05 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:1239 3.5 3:a3e59ffe:::rbd_object_map.1124f3df56b6:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T09:07:31.391 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:21.518346+0000", 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.8175318370000006, 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069812829999999996, 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1239 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.518346+0000", 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.518346+0000", 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.518346+0000", 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:31.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.518350+0000", 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9519999999999996e-06 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.518357+0000", 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0559999999999999e-06 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.518362+0000", 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.520492+0000", 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021308299999999998 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.520590+0000", 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7825999999999995e-05 2024-09-13T09:07:31.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.520995+0000", 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040503099999999998 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.523124+0000", 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021285610000000002 2024-09-13T09:07:31.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.525227+0000", 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021033879999999999 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.525281+0000", 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3424000000000002e-05 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:21.525328+0000", 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7017000000000001e-05 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:6584 3.e 3:741aa8c1:::rbd_data.11242aa72797.0000000000000024:7 [sparse-read 393216~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.785091+0000", 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.5507872049999998, 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005546881, 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6584 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.785091+0000", 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.785091+0000", 2024-09-13T09:07:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.785089+0000", 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.785102+0000", 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3376e-05 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.785146+0000", 2024-09-13T09:07:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3439999999999997e-05 2024-09-13T09:07:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.785167+0000", 2024-09-13T09:07:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-05 2024-09-13T09:07:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.788624+0000", 2024-09-13T09:07:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034574319999999999 2024-09-13T09:07:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.788843+0000", 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021922399999999999 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.790638+0000", 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001794749 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7637 3.16 3:68c1dda0:::rbd_data.11242ae5e73a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13893632~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.836681+0000", 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4991968529999999, 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015131581999999999, 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7637 2024-09-13T09:07:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.836681+0000", 2024-09-13T09:07:31.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.836681+0000", 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.836679+0000", 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:07:31.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.838596+0000", 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001917076 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.838630+0000", 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4044000000000001e-05 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.838644+0000", 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3998999999999999e-05 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.838710+0000", 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6554e-05 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.838956+0000", 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024522900000000002 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.839389+0000", 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00043297700000000003 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.851130+0000", 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011740957 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.851731+0000", 2024-09-13T09:07:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00060124699999999996 2024-09-13T09:07:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.851771+0000", 2024-09-13T09:07:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0595e-05 2024-09-13T09:07:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.851813+0000", 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1539999999999999e-05 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4388.0:7659 3.1e e30/22 3:7a598ec7:::rbd_data.11242ae5e73a.000000000000000d:head v 30'21, mlcod=29'19)", 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.872741+0000", 2024-09-13T09:07:31.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4631376390000002, 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014460859, 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.872741+0000", 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.872741+0000", 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.872739+0000", 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876082+0000", 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003343614 2024-09-13T09:07:31.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876110+0000", 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7645e-05 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876123+0000", 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2795e-05 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876166+0000", 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3182000000000001e-05 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876184+0000", 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8233e-05 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887160+0000", 2024-09-13T09:07:31.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010975446999999999 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887201+0000", 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1897999999999998e-05 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7686 3.16 3:68c1dda0:::rbd_data.11242ae5e73a.0000000000000000:head [stat,truncate 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.983675+0000", 2024-09-13T09:07:31.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.3522029829999997, 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054243269999999996, 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7686 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.983675+0000", 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.983675+0000", 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.983672+0000", 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.983687+0000", 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5512e-05 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.983729+0000", 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2004e-05 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.983745+0000", 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5780000000000001e-05 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.983846+0000", 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010088499999999999 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.984004+0000", 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000157973 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T09:07:31.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.984187+0000", 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000182666 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.987369+0000", 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003182748 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.988987+0000", 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016175180000000001 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.989052+0000", 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4885e-05 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.989100+0000", 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7840999999999998e-05 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7938 3.16 3:68c1dda0:::rbd_data.11242ae5e73a.0000000000000000:head [stat,delete] snapc 8=[8] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:25.347452+0000", 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9884266369999999, 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010723353, 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7938 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347452+0000", 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347452+0000", 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347449+0000", 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:31.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347462+0000", 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2374999999999999e-05 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347486+0000", 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4617000000000002e-05 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347496+0000", 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0210999999999999e-05 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347557+0000", 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0251999999999998e-05 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347650+0000", 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2985999999999994e-05 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.347839+0000", 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018899699999999999 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.350611+0000", 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027722749999999998 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.358083+0000", 2024-09-13T09:07:31.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074717289999999999 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.358138+0000", 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:25.358175+0000", 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7153000000000002e-05 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11327 3.e 3:71ec481d:::rbd_data.1124ea59ddac.00000000000000e2:head [set-alloc-hint object_size 1048576 write_size 1048576,write 778862~7570] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.237999+0000", 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.0978789529999999, 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005560097, 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11327 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.237999+0000", 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.237999+0000", 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.237996+0000", 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999962 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.238025+0000", 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9770000000000001e-05 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.238070+0000", 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4168999999999997e-05 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.238093+0000", 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3560000000000001e-05 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.238217+0000", 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012392600000000001 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.238564+0000", 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034708299999999998 2024-09-13T09:07:31.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.238954+0000", 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038944700000000001 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.241623+0000", 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026696200000000002 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.243320+0000", 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001696471 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.243461+0000", 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000140989 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.243559+0000", 2024-09-13T09:07:31.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8639000000000002e-05 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11739 3.5 3:a78b443e:::rbd_data.1124ea59ddac.0000000000000079:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.548525+0000", 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.7873534520000001, 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005257365, 2024-09-13T09:07:31.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11739 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548525+0000", 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548525+0000", 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548521+0000", 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548539+0000", 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7577000000000001e-05 2024-09-13T09:07:31.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548585+0000", 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5739999999999999e-05 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548597+0000", 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2224e-05 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548668+0000", 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1548e-05 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548801+0000", 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013232599999999999 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.548940+0000", 2024-09-13T09:07:31.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139166 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.549853+0000", 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091356 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.553683+0000", 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038301120000000001 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.553741+0000", 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7574000000000001e-05 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.553782+0000", 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1103999999999998e-05 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11980 3.16 3:6ff08dcf:::rbd_data.1124ea59ddac.0000000000000034:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:31.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:30.451398+0000", 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.88447980000000004, 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051469920000000004, 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11980 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.451398+0000", 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.451398+0000", 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.451397+0000", 2024-09-13T09:07:31.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.451405+0000", 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0150000000000003e-06 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.451422+0000", 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7136999999999999e-05 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.451430+0000", 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2779999999999993e-06 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.451482+0000", 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1434000000000003e-05 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.451631+0000", 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000148882 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.456545+0000", 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049148200000000003 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11989 3.16 3:6ff08dcf:::rbd_data.1124ea59ddac.0000000000000034:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:30.457151+0000", 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.87872772799999999, 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065700510000000004, 2024-09-13T09:07:31.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11989 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457151+0000", 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457151+0000", 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457149+0000", 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457155+0000", 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1599999999999997e-06 2024-09-13T09:07:31.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457169+0000", 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.448e-05 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457177+0000", 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0870000000000007e-06 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457241+0000", 2024-09-13T09:07:31.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3336999999999993e-05 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457330+0000", 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9637999999999998e-05 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.457528+0000", 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019819299999999999 2024-09-13T09:07:31.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.458161+0000", 2024-09-13T09:07:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000632305 2024-09-13T09:07:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.463671+0000", 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055098940000000004 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.463701+0000", 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0713999999999998e-05 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.463721+0000", 2024-09-13T09:07:31.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9250999999999999e-05 2024-09-13T09:07:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11998 3.16 3:697432af:::rbd_data.1124ea59ddac.0000000000000044:head [delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:31.455 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:30.466885+0000", 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.86899331499999999, 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068616880000000003, 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11998 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.466885+0000", 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.466885+0000", 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.466883+0000", 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.466892+0000", 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5010000000000004e-06 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.466914+0000", 2024-09-13T09:07:31.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2011999999999999e-05 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.466924+0000", 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.006e-05 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.466972+0000", 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8072999999999999e-05 2024-09-13T09:07:31.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.467067+0000", 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5027999999999999e-05 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.467254+0000", 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018698799999999999 2024-09-13T09:07:31.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468674+0000", 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014198920000000001 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.473642+0000", 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004968737 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.473702+0000", 2024-09-13T09:07:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9546000000000002e-05 2024-09-13T09:07:31.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.473747+0000", 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4583999999999999e-05 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4388.0:12005 3.11 e35/22 3:889ec9a4:::rbd_data.1124ea59ddac.000000000000006f:head v 35'45, mlcod=35'41)", 2024-09-13T09:07:31.464 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:30.468120+0000", 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.86775833700000005, 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054949489999999998, 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468120+0000", 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:31.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468120+0000", 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468119+0000", 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468127+0000", 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4610000000000002e-06 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468136+0000", 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8120000000000002e-06 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468143+0000", 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2480000000000001e-06 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468236+0000", 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3147999999999997e-05 2024-09-13T09:07:31.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.468256+0000", 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9629000000000001e-05 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.473541+0000", 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052852699999999999 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:31.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:30.473615+0000", 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4071000000000003e-05 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:31.469 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:31.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:31.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:31.473 INFO:teuthology.orchestra.run.smithi031.stdout:579 punch from 0x1044e25 to 0x10466bb, (0x1896 bytes) 2024-09-13T09:07:31.473 INFO:teuthology.orchestra.run.smithi031.stdout:580 trunc from 0x2d127e9 to 0xebdeef6 2024-09-13T09:07:31.473 INFO:teuthology.orchestra.run.smithi031.stdout:581 read 0x2b9f6de thru 0x2baf375 (0xfc98 bytes) 2024-09-13T09:07:31.473 INFO:teuthology.orchestra.run.smithi031.stdout:582 writesame 0xd13c306 thru 0xd13d1c6 data_size 0x1(0xec1 bytes) 2024-09-13T09:07:31.473 INFO:teuthology.orchestra.run.smithi031.stdout:583 write 0x83f0710 thru 0x83fcd07 (0xc5f8 bytes) 2024-09-13T09:07:31.474 INFO:teuthology.orchestra.run.smithi031.stdout:584 punch from 0x873521 to 0x87f5a1, (0xc080 bytes) 2024-09-13T09:07:31.474 INFO:teuthology.orchestra.run.smithi031.stdout:585 punch from 0x9b4a170 to 0x9b54c53, (0xaae3 bytes) 2024-09-13T09:07:31.474 INFO:teuthology.orchestra.run.smithi031.stdout:587 write 0x7612d42 thru 0x761816b (0x542a bytes) 2024-09-13T09:07:31.474 INFO:teuthology.orchestra.run.smithi031.stdout:588 read 0x2b6a7c5 thru 0x2b6b5ac (0xde8 bytes) 2024-09-13T09:07:31.474 INFO:teuthology.orchestra.run.smithi031.stdout:590 punch from 0x9ab3834 to 0x9ac31ce, (0xf99a bytes) 2024-09-13T09:07:31.474 INFO:teuthology.orchestra.run.smithi031.stdout:592 writesame 0x16cb9e2 thru 0x16dab70 data_size 0x3(0xf18f bytes) 2024-09-13T09:07:31.475 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:31.466+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:31.503 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:31.503 INFO:teuthology.orchestra.run.smithi031.stdout:595 read 0x927c672 thru 0x928ae4f (0xe7de bytes) 2024-09-13T09:07:31.508 INFO:teuthology.orchestra.run.smithi031.stdout:596 read 0xe7ea8f2 thru 0xe7ec498 (0x1ba7 bytes) 2024-09-13T09:07:31.509 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:31.509 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:31.509 INFO:teuthology.orchestra.run.smithi031.stdout:599 write 0xd38b6df thru 0xd39111a (0x5a3c bytes) 2024-09-13T09:07:31.531 INFO:teuthology.orchestra.run.smithi031.stdout:600 write 0xed2c980 thru 0xed37b51 (0xb1d2 bytes) 2024-09-13T09:07:31.534 INFO:teuthology.orchestra.run.smithi031.stdout:601 writesame 0x8c1db12 thru 0x8c298bc data_size 0x75(0xbdab bytes) 2024-09-13T09:07:31.541 INFO:teuthology.orchestra.run.smithi031.stdout:602 write 0x883db62 thru 0x8842a32 (0x4ed1 bytes) 2024-09-13T09:07:31.543 INFO:teuthology.orchestra.run.smithi031.stdout:603 read 0x4f0cfc5 thru 0x4f105e2 (0x361e bytes) 2024-09-13T09:07:31.543 INFO:teuthology.orchestra.run.smithi031.stdout:605 write 0x7357dde thru 0x7360f3d (0x9160 bytes) 2024-09-13T09:07:31.547 INFO:teuthology.orchestra.run.smithi031.stdout:607 read 0x44f9c37 thru 0x44fa2f2 (0x6bc bytes) 2024-09-13T09:07:31.547 INFO:teuthology.orchestra.run.smithi031.stdout:609 read 0xe442dd6 thru 0xe44b3e5 (0x8610 bytes) 2024-09-13T09:07:31.547 INFO:teuthology.orchestra.run.smithi031.stdout:610 read 0x546dd8a thru 0x546ed7a (0xff1 bytes) 2024-09-13T09:07:31.548 INFO:teuthology.orchestra.run.smithi031.stdout:611 write 0x55b3273 thru 0x55b3712 (0x4a0 bytes) 2024-09-13T09:07:31.552 INFO:teuthology.orchestra.run.smithi031.stdout:612 read 0xdbe48fd thru 0xdbeef00 (0xa604 bytes) 2024-09-13T09:07:31.553 INFO:teuthology.orchestra.run.smithi031.stdout:613 write 0xe7e93bb thru 0xe7ede4c (0x4a92 bytes) 2024-09-13T09:07:31.555 INFO:teuthology.orchestra.run.smithi031.stdout:614 read 0x6513a67 thru 0x651a1dd (0x6777 bytes) 2024-09-13T09:07:31.556 INFO:teuthology.orchestra.run.smithi031.stdout:615 write 0x4004fe thru 0x409580 (0x9083 bytes) 2024-09-13T09:07:31.561 INFO:teuthology.orchestra.run.smithi031.stdout:617 read 0x2862b04 thru 0x2867d87 (0x5284 bytes) 2024-09-13T09:07:31.562 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:31.562 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:31.562 INFO:teuthology.orchestra.run.smithi031.stdout:625 read 0xd7c0f0c thru 0xd7c5cbb (0x4db0 bytes) 2024-09-13T09:07:31.563 INFO:teuthology.orchestra.run.smithi031.stdout:628 writesame 0x128e175 thru 0x128f9a2 data_size 0x5(0x182e bytes) 2024-09-13T09:07:31.567 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:31.567 INFO:teuthology.orchestra.run.smithi031.stdout:630 read 0xbc4d558 thru 0xbc5314d (0x5bf6 bytes) 2024-09-13T09:07:31.567 INFO:teuthology.orchestra.run.smithi031.stdout:632 punch from 0x526c982 to 0x5274010, (0x768e bytes) 2024-09-13T09:07:31.567 INFO:teuthology.orchestra.run.smithi031.stdout:633 punch from 0x6842605 to 0x684386f, (0x126a bytes) 2024-09-13T09:07:31.567 INFO:teuthology.orchestra.run.smithi031.stdout:634 clone 8 order 24 su 65536 sc 2 2024-09-13T09:07:31.572 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:31.568+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:31.639 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:31.639 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:31.640 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:31.640 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:31.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:31.659 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:31.659 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:31.674 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:31.670+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:31.675 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:31.676 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:31.692 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:31.692 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:31.708 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:31.709 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:31.729 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:31.729 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:31.749 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:31.750 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:31.771 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:31.771 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:31.774 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:31.774 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:31.774 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:31.772+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:31.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:31.792 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:31.875 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:31.872+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:31.907 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:31.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:31.977 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:31.972+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:32.056 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:32.056 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:32.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:32.116 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:32.114+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:32.178 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:32.172+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:32.196 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:32.196 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:32.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:32.246 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:32.247 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:07:32.247 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:07:32.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:07:32.295 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:32.292+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:32.382 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:32.378+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:32.384 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:32.384 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:32.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:32.452 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone7 from 0xed37b52 (overlap 0x2d127e9) to 0x0 2024-09-13T09:07:32.483 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:32.480+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:32.581 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:32.581 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:32.583 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:32.580+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 37, 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T09:06:44.149006+0000", 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T09:07:32.252646+0000", 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T09:06:55.611487+0000", 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T09:07:27.209290+0000", 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5804032, 2024-09-13T09:07:32.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T09:07:32.596 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:06:55.131330+0000", 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T09:07:32.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T09:07:32.598 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:07:32.599 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:07:32.600 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:07:32.601 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:07:32.602 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:02.743102+0000", 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:07:32.604 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:07:32.605 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:07:32.606 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:07:32.607 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:07:32.608 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9800000190734863, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9800000190734863, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:07:32.609 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 4.380000114440918, 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 4.380000114440918, 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:12.517158+0000", 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:07:32.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T09:07:32.611 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "37", 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:07:32.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 11, 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 37, 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:07:32.613 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:07:32.614 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:07:32.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.7400000095367432, 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.7400000095367432, 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:32.616 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "33c6fc7d-9828-4500-b103-b96d1cd7f420", 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:32.617 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.618 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:32.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.620 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6818/3291756747", 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6822/3291756747", 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6830/3291756747", 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6826/3291756747", 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:32.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "869ea3c5-8e6e-4b49-b70e-083f242a7aae", 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:32.623 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:32.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.625 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/533520566", 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/533520566", 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/533520566", 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/533520566", 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:32.628 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e144d496-76ee-46e0-bdc6-135c77128535", 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:32.629 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T09:07:32.630 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.631 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6819/2253604482", 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6823/2253604482", 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/2253604482", 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6827/2253604482", 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:32.634 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "696b9414-9c19-4984-a8ff-14548e46b357", 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:32.635 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T09:07:32.636 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.637 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.638 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.639 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/3367214693", 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/3367214693", 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/3367214693", 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/3367214693", 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ac58040d-9b54-42ca-b143-1816abb54816", 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:32.641 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6800", 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6801", 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:32.642 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6802", 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6803", 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6806", 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6807", 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:32.644 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6804", 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6805", 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6801/3778074750", 2024-09-13T09:07:32.645 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6803/3778074750", 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6807/3778074750", 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6805/3778074750", 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2", 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:32.646 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 35, 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6808", 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6809", 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6811", 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6813", 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6819", 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:32.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6824", 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6815", 2024-09-13T09:07:32.650 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6817", 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6809/1926603087", 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6813/1926603087", 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6824/1926603087", 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6817/1926603087", 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:32.651 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d", 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:32.652 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6820", 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6821", 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6823", 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6825", 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6830", 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6831", 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6827", 2024-09-13T09:07:32.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6828", 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6821/766305500", 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6825/766305500", 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6831/766305500", 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6828/766305500", 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:32.657 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f36c332c-9497-47e8-b943-f6b6dcaebeb8", 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:07:32.658 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6810", 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6812", 2024-09-13T09:07:32.659 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6814", 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.660 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6816", 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6826", 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6829", 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6818", 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:32.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6822", 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6812/1032956039", 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6816/1032956039", 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6829/1032956039", 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6822/1032956039", 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:07:32.663 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:32.664 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.183436+0000", 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.264080+0000", 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.253587+0000", 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:32.666 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.244429+0000", 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.369458+0000", 2024-09-13T09:07:32.667 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.158089+0000", 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.668 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.217295+0000", 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:07:32.669 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.332254+0000", 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T09:07:32.670 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T09:07:32.671 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:32.672 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:32.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:32.675 INFO:tasks.thrashosds.thrasher:j is {'epoch': 37, 'fsid': 'fe16bf6d-4166-4417-80fd-1123617a33d3', 'created': '2024-09-13T09:06:44.149006+0000', 'modified': '2024-09-13T09:07:32.252646+0000', 'last_up_change': '2024-09-13T09:06:55.611487+0000', 'last_in_change': '2024-09-13T09:07:27.209290+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:06:55.131330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:07:02.743102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:07:12.517158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 11, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '33c6fc7d-9828-4500-b103-b96d1cd7f420', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6818', 'nonce': 3291756747}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6822', 'nonce': 3291756747}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6830', 'nonce': 3291756747}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6826', 'nonce': 3291756747}]}, 'public_addr': '172.21.15.31:6818/3291756747', 'cluster_addr': '172.21.15.31:6822/3291756747', 'heartbeat_back_addr': '172.21.15.31:6830/3291756747', 'heartbeat_front_addr': '172.21.15.31:6826/3291756747', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 533520566}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 533520566}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 533520566}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 533520566}]}, 'public_addr': '172.21.15.31:6801/533520566', 'cluster_addr': '172.21.15.31:6803/533520566', 'heartbeat_back_addr': '172.21.15.31:6807/533520566', 'heartbeat_front_addr': '172.21.15.31:6805/533520566', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e144d496-76ee-46e0-bdc6-135c77128535', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6817', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2253604482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6821', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2253604482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6829', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 2253604482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6825', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 2253604482}]}, 'public_addr': '172.21.15.31:6819/2253604482', 'cluster_addr': '172.21.15.31:6823/2253604482', 'heartbeat_back_addr': '172.21.15.31:6831/2253604482', 'heartbeat_front_addr': '172.21.15.31:6827/2253604482', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '696b9414-9c19-4984-a8ff-14548e46b357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 3367214693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 3367214693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 3367214693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 3367214693}]}, 'public_addr': '172.21.15.31:6809/3367214693', 'cluster_addr': '172.21.15.31:6811/3367214693', 'heartbeat_back_addr': '172.21.15.31:6815/3367214693', 'heartbeat_front_addr': '172.21.15.31:6813/3367214693', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ac58040d-9b54-42ca-b143-1816abb54816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6800', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6801', 'nonce': 3778074750}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 3778074750}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 3778074750}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 3778074750}]}, 'public_addr': '172.21.15.60:6801/3778074750', 'cluster_addr': '172.21.15.60:6803/3778074750', 'heartbeat_back_addr': '172.21.15.60:6807/3778074750', 'heartbeat_front_addr': '172.21.15.60:6805/3778074750', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1926603087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6811', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 1926603087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6819', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6824', 'nonce': 1926603087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6815', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 1926603087}]}, 'public_addr': '172.21.15.60:6809/1926603087', 'cluster_addr': '172.21.15.60:6813/1926603087', 'heartbeat_back_addr': '172.21.15.60:6824/1926603087', 'heartbeat_front_addr': '172.21.15.60:6817/1926603087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 766305500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6823', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 766305500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 766305500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6827', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6828', 'nonce': 766305500}]}, 'public_addr': '172.21.15.60:6821/766305500', 'cluster_addr': '172.21.15.60:6825/766305500', 'heartbeat_back_addr': '172.21.15.60:6831/766305500', 'heartbeat_front_addr': '172.21.15.60:6828/766305500', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f36c332c-9497-47e8-b943-f6b6dcaebeb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6812', 'nonce': 1032956039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6816', 'nonce': 1032956039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 1032956039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6822', 'nonce': 1032956039}]}, 'public_addr': '172.21.15.60:6812/1032956039', 'cluster_addr': '172.21.15.60:6816/1032956039', 'heartbeat_back_addr': '172.21.15.60:6829/1032956039', 'heartbeat_front_addr': '172.21.15.60:6822/1032956039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.183436+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.264080+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.253587+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.244429+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.369458+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.158089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.217295+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.332254+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:07:32.675 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-13T09:07:32.682 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-13T09:07:32.684 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:32.680+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:32.785 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:32.780+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:32.801 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:32.801 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:32.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:32.854 INFO:teuthology.orchestra.run.smithi031.stdout:635 trunc from 0xed37b52 to 0x59a27c8 2024-09-13T09:07:32.886 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:32.882+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:32.926 INFO:teuthology.orchestra.run.smithi031.stdout:636 trunc from 0x59a27c8 to 0xccbba8a 2024-09-13T09:07:32.949 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:32.949 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:32.951 INFO:teuthology.orchestra.run.smithi031.stdout:637 write 0xe7912ca thru 0xe79451f (0x3256 bytes) 2024-09-13T09:07:32.957 INFO:teuthology.orchestra.run.smithi031.stdout:638 read 0x22ea5e0 thru 0x22ebddb (0x17fc bytes) 2024-09-13T09:07:32.958 INFO:teuthology.orchestra.run.smithi031.stdout:639 writesame 0x30bc1f8 thru 0x30c697d data_size 0x2(0xa786 bytes) 2024-09-13T09:07:32.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:32.987 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:32.984+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:33.061 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.061 INFO:teuthology.orchestra.run.smithi031.stdout:641 write 0x83fbdae thru 0x8406754 (0xa9a7 bytes) 2024-09-13T09:07:33.067 INFO:teuthology.orchestra.run.smithi031.stdout:642 write 0x7f442d8 thru 0x7f46a9e (0x27c7 bytes) 2024-09-13T09:07:33.073 INFO:teuthology.orchestra.run.smithi031.stdout:644 write 0x69a0594 thru 0x69a6ef7 (0x6964 bytes) 2024-09-13T09:07:33.077 INFO:teuthology.orchestra.run.smithi031.stdout:645 write 0x640ce03 thru 0x6413818 (0x6a16 bytes) 2024-09-13T09:07:33.083 INFO:teuthology.orchestra.run.smithi031.stdout:648 writesame 0x6bda37e thru 0x6be1798 data_size 0x1(0x741b bytes) 2024-09-13T09:07:33.087 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:33.084+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:33.137 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:33.138 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:33.138 INFO:teuthology.orchestra.run.smithi031.stdout:650 writesame 0xa3ce239 thru 0xa3cf00d data_size 0x1(0xdd5 bytes) 2024-09-13T09:07:33.148 INFO:teuthology.orchestra.run.smithi031.stdout:651 write 0x93a0f50 thru 0x93ab949 (0xa9fa bytes) 2024-09-13T09:07:33.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:33.154 INFO:teuthology.orchestra.run.smithi031.stdout:653 writesame 0x479c578 thru 0x47a4982 data_size 0xb(0x840b bytes) 2024-09-13T09:07:33.160 INFO:teuthology.orchestra.run.smithi031.stdout:655 write 0xbf8bd88 thru 0xbf8e427 (0x26a0 bytes) 2024-09-13T09:07:33.164 INFO:teuthology.orchestra.run.smithi031.stdout:656 read 0x2413f89 thru 0x241eeb1 (0xaf29 bytes) 2024-09-13T09:07:33.166 INFO:teuthology.orchestra.run.smithi031.stdout:658 write 0x6743857 thru 0x674c0ef (0x8899 bytes) 2024-09-13T09:07:33.169 INFO:teuthology.orchestra.run.smithi031.stdout:662 writesame 0x52480b5 thru 0x525691e data_size 0x47(0xe86a bytes) 2024-09-13T09:07:33.173 INFO:teuthology.orchestra.run.smithi031.stdout:663 read 0xbfb65b thru 0xc0b30d (0xfcb3 bytes) 2024-09-13T09:07:33.175 INFO:teuthology.orchestra.run.smithi031.stdout:665 read 0x92ffd98 thru 0x93092a1 (0x950a bytes) 2024-09-13T09:07:33.200 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:33.198+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:33.203 INFO:teuthology.orchestra.run.smithi031.stdout:668 trunc from 0xe794520 to 0x40c4950 2024-09-13T09:07:33.233 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.249 INFO:teuthology.orchestra.run.smithi031.stdout:672 write 0x589f27d thru 0x58a88c2 (0x9646 bytes) 2024-09-13T09:07:33.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:07:33.283 INFO:teuthology.orchestra.run.smithi031.stdout:673 write 0xc6fcf9f thru 0xc7040c0 (0x7122 bytes) 2024-09-13T09:07:33.285 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:33.285 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:33.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:33.315 INFO:teuthology.orchestra.run.smithi031.stdout:674 trunc from 0xc7040c1 to 0x22b4297 2024-09-13T09:07:33.315 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:33.312+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:33.336 INFO:teuthology.orchestra.run.smithi031.stdout:675 punch from 0x9784f0 to 0x979042, (0xb52 bytes) 2024-09-13T09:07:33.336 INFO:teuthology.orchestra.run.smithi031.stdout:676 read 0x1f600dc thru 0x1f695d4 (0x94f9 bytes) 2024-09-13T09:07:33.337 INFO:teuthology.orchestra.run.smithi031.stdout:677 read 0x146dd30 thru 0x1475cdd (0x7fae bytes) 2024-09-13T09:07:33.339 INFO:teuthology.orchestra.run.smithi031.stdout:678 punch from 0x102f8bd to 0x103903d, (0x9780 bytes) 2024-09-13T09:07:33.349 INFO:teuthology.orchestra.run.smithi031.stdout:679 writesame 0x2913e25 thru 0x2919b4f data_size 0x11(0x5d2b bytes) 2024-09-13T09:07:33.351 INFO:teuthology.orchestra.run.smithi031.stdout:680 writesame 0x20e03e5 thru 0x20e4cc6 data_size 0x13(0x48e2 bytes) 2024-09-13T09:07:33.356 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.356 INFO:teuthology.orchestra.run.smithi031.stdout:682 read 0x1616b51 thru 0x161bce5 (0x5195 bytes) 2024-09-13T09:07:33.357 INFO:teuthology.orchestra.run.smithi031.stdout:683 read 0x2f7031 thru 0x2fffeb (0x8fbb bytes) 2024-09-13T09:07:33.385 INFO:teuthology.orchestra.run.smithi031.stdout:684 write 0xa0540ee thru 0xa0619f0 (0xd903 bytes) 2024-09-13T09:07:33.389 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:33.384+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:33.392 INFO:teuthology.orchestra.run.smithi031.stdout:685 write 0x5a6149a thru 0x5a71015 (0xfb7c bytes) 2024-09-13T09:07:33.404 INFO:teuthology.orchestra.run.smithi031.stdout:687 trunc from 0xa0619f1 to 0x8094685 2024-09-13T09:07:33.427 INFO:teuthology.orchestra.run.smithi031.stdout:688 trunc from 0x8094685 to 0x20a2d66 2024-09-13T09:07:33.459 INFO:teuthology.orchestra.run.smithi031.stdout:689 writesame 0x64c1791 thru 0x64c620c data_size 0x1c(0x4a7c bytes) 2024-09-13T09:07:33.474 INFO:teuthology.orchestra.run.smithi031.stdout:690 write 0xa08b8c1 thru 0xa092d81 (0x74c1 bytes) 2024-09-13T09:07:33.482 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:33.482 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:33.489 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:33.484+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:33.493 INFO:teuthology.orchestra.run.smithi031.stdout:692 write 0xacbc684 thru 0xacc58a6 (0x9223 bytes) 2024-09-13T09:07:33.496 INFO:teuthology.orchestra.run.smithi031.stdout:693 write 0x2262634 thru 0x226d8b7 (0xb284 bytes) 2024-09-13T09:07:33.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:33.499 INFO:teuthology.orchestra.run.smithi031.stdout:694 write 0x43e67c0 thru 0x43ee1c9 (0x7a0a bytes) 2024-09-13T09:07:33.504 INFO:teuthology.orchestra.run.smithi031.stdout:696 read 0x4a05ab3 thru 0x4a0f0ed (0x963b bytes) 2024-09-13T09:07:33.509 INFO:teuthology.orchestra.run.smithi031.stdout:698 read 0x7dc270f thru 0x7dcc972 (0xa264 bytes) 2024-09-13T09:07:33.509 INFO:teuthology.orchestra.run.smithi031.stdout:702 read 0x6553d17 thru 0x6561b4c (0xde36 bytes) 2024-09-13T09:07:33.509 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.509 INFO:teuthology.orchestra.run.smithi031.stdout:705 punch from 0xc59f3a to 0xc660dc, (0xc1a2 bytes) 2024-09-13T09:07:33.518 INFO:teuthology.orchestra.run.smithi031.stdout:706 write 0xc11ea9d thru 0xc11f448 (0x9ac bytes) 2024-09-13T09:07:33.526 INFO:teuthology.orchestra.run.smithi031.stdout:708 trunc from 0xc11f449 to 0xa6382df 2024-09-13T09:07:33.560 INFO:teuthology.orchestra.run.smithi031.stdout:709 writesame 0xc1c6981 thru 0xc1c76a9 data_size 0x1(0xd29 bytes) 2024-09-13T09:07:33.560 INFO:teuthology.orchestra.run.smithi031.stdout:710 write 0x4e915cc thru 0x4e96bdb (0x5610 bytes) 2024-09-13T09:07:33.565 INFO:teuthology.orchestra.run.smithi031.stdout:711 write 0x464947f thru 0x4658cdc (0xf85e bytes) 2024-09-13T09:07:33.568 INFO:teuthology.orchestra.run.smithi031.stdout:712 punch from 0x69c1ea9 to 0x69d116c, (0xf2c3 bytes) 2024-09-13T09:07:33.586 INFO:teuthology.orchestra.run.smithi031.stdout:714 trunc from 0xc1c76aa to 0x78d53f 2024-09-13T09:07:33.591 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:33.586+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:33.630 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:33.630 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:33.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:33.737 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:33.734+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:33.771 INFO:teuthology.orchestra.run.smithi031.stdout:715 trunc from 0x78d53f to 0xfc24a7 2024-09-13T09:07:33.792 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:33.790+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:33.809 INFO:teuthology.orchestra.run.smithi031.stdout:716 trunc from 0xfc24a7 to 0xc113397 2024-09-13T09:07:33.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:33.813 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:33.813 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:33.821 INFO:teuthology.orchestra.run.smithi031.stdout:717 read 0x54841ed thru 0x549376d (0xf581 bytes) 2024-09-13T09:07:33.822 INFO:teuthology.orchestra.run.smithi031.stdout:718 write 0x6ba945b thru 0x6bb51dd (0xbd83 bytes) 2024-09-13T09:07:33.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:33.829 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.831 INFO:teuthology.orchestra.run.smithi031.stdout:720 trunc from 0xc113397 to 0xad82cfa 2024-09-13T09:07:33.854 INFO:teuthology.orchestra.run.smithi031.stdout:721 writesame 0xe465291 thru 0xe468f1d data_size 0x1(0x3c8d bytes) 2024-09-13T09:07:33.854 INFO:teuthology.orchestra.run.smithi031.stdout:722 read 0x3294260 thru 0x329bfa6 (0x7d47 bytes) 2024-09-13T09:07:33.854 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.854 INFO:teuthology.orchestra.run.smithi031.stdout:725 write 0xc293cef thru 0xc299e41 (0x6153 bytes) 2024-09-13T09:07:33.859 INFO:teuthology.orchestra.run.smithi031.stdout:728 read 0xb1ff3af thru 0xb20e164 (0xedb6 bytes) 2024-09-13T09:07:33.859 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.876 INFO:teuthology.orchestra.run.smithi031.stdout:730 trunc from 0xe468f1e to 0x4aac433 2024-09-13T09:07:33.892 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.892 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:33.888+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:33.907 INFO:teuthology.orchestra.run.smithi031.stdout:732 writesame 0x869cc9f thru 0x869eda5 data_size 0x13(0x2107 bytes) 2024-09-13T09:07:33.907 INFO:teuthology.orchestra.run.smithi031.stdout:734 punch from 0x7a8573d to 0x7a91b54, (0xc417 bytes) 2024-09-13T09:07:33.908 INFO:teuthology.orchestra.run.smithi031.stdout:735 flatten 2024-09-13T09:07:33.926 INFO:teuthology.orchestra.run.smithi031.stdout:736 trunc from 0x869eda6 to 0x52d2a7e 2024-09-13T09:07:33.957 INFO:teuthology.orchestra.run.smithi031.stdout:737 write 0x9e1bbae thru 0x9e2929b (0xd6ee bytes) 2024-09-13T09:07:33.965 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:33.965 INFO:teuthology.orchestra.run.smithi031.stdout:739 punch from 0x815d687 to 0x8163463, (0x5ddc bytes) 2024-09-13T09:07:33.965 INFO:teuthology.orchestra.run.smithi031.stdout:740 read 0x7ed702c thru 0x7ed9bbc (0x2b91 bytes) 2024-09-13T09:07:33.965 INFO:teuthology.orchestra.run.smithi031.stdout:742 read 0x7b06636 thru 0x7b10925 (0xa2f0 bytes) 2024-09-13T09:07:33.974 INFO:teuthology.orchestra.run.smithi031.stdout:743 write 0xb3f857d thru 0xb407d9c (0xf820 bytes) 2024-09-13T09:07:33.981 INFO:teuthology.orchestra.run.smithi031.stdout:745 punch from 0x9611f2 to 0x9648f1, (0x36ff bytes) 2024-09-13T09:07:33.982 INFO:teuthology.orchestra.run.smithi031.stdout:746 trunc from 0xb407d9d to 0xc715b09 2024-09-13T09:07:33.984 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:33.984 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:33.989 INFO:teuthology.orchestra.run.smithi031.stdout:749 read 0x9efa35 thru 0x9fdf82 (0xe54e bytes) 2024-09-13T09:07:33.990 INFO:teuthology.orchestra.run.smithi031.stdout:750 write 0x21f870a thru 0x2208160 (0xfa57 bytes) 2024-09-13T09:07:33.992 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:33.988+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:33.997 INFO:teuthology.orchestra.run.smithi031.stdout:751 write 0xc3cc775 thru 0xc3d0c7d (0x4509 bytes) 2024-09-13T09:07:34.002 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:34.002 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:34.008 INFO:teuthology.orchestra.run.smithi031.stdout:752 trunc from 0xc715b09 to 0x90ac59a 2024-09-13T09:07:34.013 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:34.013 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:34.019 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:34.019 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:34.031 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:34.037 INFO:teuthology.orchestra.run.smithi031.stdout:753 writesame 0x9c65575 thru 0x9c6c41e data_size 0x5(0x6eaa bytes) 2024-09-13T09:07:34.038 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:34.038 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:34.039 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:34.039 INFO:teuthology.orchestra.run.smithi031.stdout:755 read 0x102f14d thru 0x103bb5a (0xca0e bytes) 2024-09-13T09:07:34.039 INFO:teuthology.orchestra.run.smithi031.stdout:758 writesame 0x8aa6554 thru 0x8aad805 data_size 0x2(0x72b2 bytes) 2024-09-13T09:07:34.050 INFO:teuthology.orchestra.run.smithi031.stdout:759 write 0x43aaa38 thru 0x43b6cd0 (0xc299 bytes) 2024-09-13T09:07:34.057 INFO:teuthology.orchestra.run.smithi031.stdout:760 write 0xd2490c thru 0xd2ecec (0xa3e1 bytes) 2024-09-13T09:07:34.058 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:34.058 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:34.061 INFO:teuthology.orchestra.run.smithi031.stdout:762 write 0x3f3a5ec thru 0x3f45cbd (0xb6d2 bytes) 2024-09-13T09:07:34.063 INFO:teuthology.orchestra.run.smithi031.stdout:765 write 0x3e28aee thru 0x3e2f580 (0x6a93 bytes) 2024-09-13T09:07:34.065 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:34.065 INFO:teuthology.orchestra.run.smithi031.stdout:767 write 0x1025b93 thru 0x10268d7 (0xd45 bytes) 2024-09-13T09:07:34.067 INFO:teuthology.orchestra.run.smithi031.stdout:769 punch from 0x94b1bf6 to 0x94bc6d5, (0xaadf bytes) 2024-09-13T09:07:34.077 INFO:teuthology.orchestra.run.smithi031.stdout:770 write 0xbb498b2 thru 0xbb59636 (0xfd85 bytes) 2024-09-13T09:07:34.078 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:34.078 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:34.091 INFO:teuthology.orchestra.run.smithi031.stdout:771 write 0xc64c10b thru 0xc657a1c (0xb912 bytes) 2024-09-13T09:07:34.093 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:34.090+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:34.097 INFO:teuthology.orchestra.run.smithi031.stdout:773 punch from 0x47bc5f7 to 0x47c8fb4, (0xc9bd bytes) 2024-09-13T09:07:34.097 INFO:teuthology.orchestra.run.smithi031.stdout:774 write 0x469625f thru 0x46a5cee (0xfa90 bytes) 2024-09-13T09:07:34.100 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:34.100 INFO:teuthology.orchestra.run.smithi031.stdout:775 write 0x36a0d65 thru 0x36a15b7 (0x853 bytes) 2024-09-13T09:07:34.100 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:34.102 INFO:teuthology.orchestra.run.smithi031.stdout:777 read 0xc2558e7 thru 0xc25ee67 (0x9581 bytes) 2024-09-13T09:07:34.102 INFO:teuthology.orchestra.run.smithi031.stdout:778 write 0x2c348dd thru 0x2c39623 (0x4d47 bytes) 2024-09-13T09:07:34.104 INFO:teuthology.orchestra.run.smithi031.stdout:779 writesame 0x7a94303 thru 0x7a94776 data_size 0x14(0x474 bytes) 2024-09-13T09:07:34.116 INFO:teuthology.orchestra.run.smithi031.stdout:780 write 0xd5f90cc thru 0xd600f91 (0x7ec6 bytes) 2024-09-13T09:07:34.121 INFO:teuthology.orchestra.run.smithi031.stdout:782 writesame 0x74644af thru 0x746cc83 data_size 0x43(0x87d5 bytes) 2024-09-13T09:07:34.121 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:34.121 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:34.122 INFO:teuthology.orchestra.run.smithi031.stdout:783 write 0x3258194 thru 0x326273d (0xa5aa bytes) 2024-09-13T09:07:34.124 INFO:teuthology.orchestra.run.smithi031.stdout:784 read 0x454efc3 thru 0x45559ec (0x6a2a bytes) 2024-09-13T09:07:34.125 INFO:teuthology.orchestra.run.smithi031.stdout:785 write 0x3018569 thru 0x3024b8c (0xc624 bytes) 2024-09-13T09:07:34.127 INFO:teuthology.orchestra.run.smithi031.stdout:786 clone 9 order 22 su 16384 sc 3 2024-09-13T09:07:34.156 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:34.156 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:34.156 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:34.156 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:34.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:34.195 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:34.190+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:34.289 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:07:34.295 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:34.292+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:34.338 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:34.338 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:34.339 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:34.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:34.396 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:34.390+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:34.497 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:34.494+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:34.527 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:34.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11541 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1490944~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314776+0000", 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2118896240000003, 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028398428, 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11541 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314776+0000", 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314776+0000", 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314775+0000", 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314778+0000", 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6029999999999999e-06 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314783+0000", 2024-09-13T09:07:34.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4500000000000003e-06 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314789+0000", 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1619999999999999e-06 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.342749+0000", 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027960137999999999 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.342840+0000", 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1609999999999999e-05 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343174+0000", 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033374499999999998 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.535 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11542 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1638400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314798+0000", 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2118668680000004, 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028618411, 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11542 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314798+0000", 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314798+0000", 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314798+0000", 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314801+0000", 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2490000000000001e-06 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314808+0000", 2024-09-13T09:07:34.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8140000000000003e-06 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314813+0000", 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2399999999999998e-06 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343221+0000", 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028407452 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343306+0000", 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5767999999999996e-05 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343417+0000", 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110357 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11543 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1785856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314817+0000", 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2118480219999999, 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028867417999999999, 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11543 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314817+0000", 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314817+0000", 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314817+0000", 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314819+0000", 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314825+0000", 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6350000000000003e-06 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314829+0000", 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4020000000000004e-06 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343480+0000", 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028650268999999999 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343569+0000", 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9732000000000001e-05 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343685+0000", 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115275 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11544 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314834+0000", 2024-09-13T09:07:34.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2118316159999996, 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029327243999999999, 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11544 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314834+0000", 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314834+0000", 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314833+0000", 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314836+0000", 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3180000000000001e-06 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314842+0000", 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9649999999999996e-06 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314846+0000", 2024-09-13T09:07:34.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8590000000000004e-06 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343727+0000", 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.028880985000000001 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.343815+0000", 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7590999999999995e-05 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344161+0000", 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034583500000000001 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11545 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2080768~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314851+0000", 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2118144730000004, 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029748916, 2024-09-13T09:07:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11545 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314851+0000", 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314851+0000", 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314850+0000", 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314853+0000", 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7099999999999999e-06 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314859+0000", 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3640000000000004e-06 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314864+0000", 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6550000000000003e-06 2024-09-13T09:07:34.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344193+0000", 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029328913000000002 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344280+0000", 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7435000000000006e-05 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344600+0000", 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031923699999999999 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11546 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2228224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314868+0000", 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2117970639999998, 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030015035999999998, 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11546 2024-09-13T09:07:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314868+0000", 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314868+0000", 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314868+0000", 2024-09-13T09:07:34.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314870+0000", 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5289999999999998e-06 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314878+0000", 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1250000000000004e-06 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314882+0000", 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7029999999999997e-06 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344650+0000", 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.029767709999999999 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344764+0000", 2024-09-13T09:07:34.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011393 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344883+0000", 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011937100000000001 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11547 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2375680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314887+0000", 2024-09-13T09:07:34.560 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.211778325, 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030498834999999998, 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11547 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314887+0000", 2024-09-13T09:07:34.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314887+0000", 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314887+0000", 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314889+0000", 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9750000000000001e-06 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314892+0000", 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8310000000000003e-06 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314895+0000", 2024-09-13T09:07:34.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.914e-06 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.344921+0000", 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030025738 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345009+0000", 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8160999999999998e-05 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345386+0000", 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00037646899999999999 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11548 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314898+0000", 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2117675270000001, 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030745146000000001, 2024-09-13T09:07:34.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11548 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314898+0000", 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314898+0000", 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314898+0000", 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314899+0000", 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5209999999999999e-06 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314902+0000", 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1109999999999999e-06 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314906+0000", 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2780000000000004e-06 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345434+0000", 2024-09-13T09:07:34.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030527110999999999 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345518+0000", 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4875000000000003e-05 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345643+0000", 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012445699999999999 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11549 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2670592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314909+0000", 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2117565299999997, 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031201804999999999, 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11549 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314909+0000", 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314909+0000", 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314908+0000", 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314910+0000", 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.46e-06 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314913+0000", 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314916+0000", 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6900000000000001e-06 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345679+0000", 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.030763092999999998 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.345767+0000", 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8197999999999998e-05 2024-09-13T09:07:34.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346111+0000", 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034368300000000001 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11550 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2818048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314918+0000", 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2117473680000002, 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031725077999999997, 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11550 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314918+0000", 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314918+0000", 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314918+0000", 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314919+0000", 2024-09-13T09:07:34.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.367e-06 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314922+0000", 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0259999999999999e-06 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314925+0000", 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8590000000000001e-06 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346204+0000", 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031279080000000001 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346298+0000", 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3653999999999995e-05 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346643+0000", 2024-09-13T09:07:34.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00034530100000000001 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11551 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314927+0000", 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2117382079999999, 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031961214000000002, 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11551 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314927+0000", 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314927+0000", 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314927+0000", 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314928+0000", 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4050000000000001e-06 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314931+0000", 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9620000000000002e-06 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314937+0000", 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0340000000000002e-06 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346675+0000", 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031737994999999998 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346779+0000", 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010354800000000001 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346888+0000", 2024-09-13T09:07:34.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109472 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11552 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314940+0000", 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2117257449999999, 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032564320000000001, 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11552 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314940+0000", 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314940+0000", 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314939+0000", 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314941+0000", 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4440000000000001e-06 2024-09-13T09:07:34.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314944+0000", 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9189999999999999e-06 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314946+0000", 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6029999999999999e-06 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.346940+0000", 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031993332999999999 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.347050+0000", 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010993999999999999 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.347504+0000", 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000454308 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11553 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314948+0000", 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2117168329999997, 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033136483000000001, 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11553 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314948+0000", 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314948+0000", 2024-09-13T09:07:34.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314948+0000", 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314950+0000", 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3260000000000001e-06 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314952+0000", 2024-09-13T09:07:34.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8270000000000002e-06 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314955+0000", 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8100000000000002e-06 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.347564+0000", 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032608919 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.347722+0000", 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000157992 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348085+0000", 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036282000000000003 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11554 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3407872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314957+0000", 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2117079300000002, 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033748444000000002, 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11554 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314957+0000", 2024-09-13T09:07:34.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314957+0000", 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314957+0000", 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314976+0000", 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8783999999999998e-05 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314981+0000", 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8790000000000002e-06 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314984+0000", 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.247e-06 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348205+0000", 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03322141 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348295+0000", 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9178999999999994e-05 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348706+0000", 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041115299999999999 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11555 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3555328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314987+0000", 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2116781449999996, 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034189124000000001, 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11555 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314987+0000", 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314987+0000", 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314987+0000", 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314988+0000", 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5349999999999999e-06 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314991+0000", 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0139999999999999e-06 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314994+0000", 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9009999999999998e-06 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348778+0000", 2024-09-13T09:07:34.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033783852000000003 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.348846+0000", 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7866000000000004e-05 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.349176+0000", 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033037000000000002 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11556 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3702784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.314996+0000", 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2116688519999999, 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035061971999999997, 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11556 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314996+0000", 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314996+0000", 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314996+0000", 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.314998+0000", 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5149999999999999e-06 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315001+0000", 2024-09-13T09:07:34.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3510000000000002e-06 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315005+0000", 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9670000000000003e-06 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.349229+0000", 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034223954000000001 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.349322+0000", 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3006999999999999e-05 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350058+0000", 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00073638299999999996 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11557 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3850240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.315009+0000", 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.211656295, 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035481024, 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11557 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315009+0000", 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315009+0000", 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315009+0000", 2024-09-13T09:07:34.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315021+0000", 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315028+0000", 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0600000000000002e-06 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315033+0000", 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350093+0000", 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035059923999999999 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350202+0000", 2024-09-13T09:07:34.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010902499999999999 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350490+0000", 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000287659 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11558 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 3997696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.315039+0000", 2024-09-13T09:07:34.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2116265349999997, 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037167545000000003, 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11558 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315039+0000", 2024-09-13T09:07:34.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315039+0000", 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315038+0000", 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315041+0000", 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.779e-06 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315047+0000", 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.28e-06 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315052+0000", 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8810000000000004e-06 2024-09-13T09:07:34.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350529+0000", 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035476925999999999 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.350596+0000", 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6698000000000004e-05 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.352206+0000", 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001610453 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:11559 3.15 3:abda0e6f:::rbd_data.1124fee72b32.0000000000000001:9 [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e34)", 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:29.315057+0000", 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2116083289999997, 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037313261, 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:34.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11559 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315057+0000", 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315057+0000", 2024-09-13T09:07:34.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315057+0000", 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315059+0000", 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.458e-06 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315065+0000", 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8159999999999999e-06 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.315077+0000", 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.256e-05 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.352236+0000", 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.037158785999999999 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.352300+0000", 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3615999999999995e-05 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:29.352370+0000", 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0427999999999997e-05 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:15159 3.c 3:33b101a6:::rbd_data.11242e26750a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15794176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.232205574, 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052673589999999999, 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 15159 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294458+0000", 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:34.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302503+0000", 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080450420000000005 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302534+0000", 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1352000000000001e-05 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302550+0000", 2024-09-13T09:07:34.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5511000000000001e-05 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302617+0000", 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7722e-05 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302628+0000", 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0997e-05 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304649+0000", 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020203460000000001 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304734+0000", 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4839999999999994e-05 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.305520+0000", 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078672300000000002 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.343824+0000", 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038304108000000003 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347049+0000", 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032249399999999999 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347102+0000", 2024-09-13T09:07:34.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2163000000000002e-05 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347133+0000", 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1674999999999997e-05 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:34.628 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:34.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:34.630 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-13T09:07:34.631 INFO:teuthology.orchestra.run.smithi031.stdout:788 write 0xd7d20b1 thru 0xd7e01c6 (0xe116 bytes) 2024-09-13T09:07:34.631 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:34.624+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:34.731 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:34.728+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:34.741 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:34.741 INFO:teuthology.orchestra.run.smithi031.stdout:790 writesame 0x3fd63c5 thru 0x3fddbf0 data_size 0x2(0x782c bytes) 2024-09-13T09:07:34.809 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:34.809 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:34.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:34.867 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:34.864+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:34.932 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:34.930+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:34.995 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:34.995 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:34.995 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:34.995 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:34.995 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:35.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:35.055 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:35.052+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:35.134 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:35.132+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:35.153 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.456399083000001, 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840076, 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 34 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694022+0000", 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694033+0000", 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.116e-05 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694051+0000", 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694065+0000", 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3716e-05 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700273+0000", 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062082320000000002 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700405+0000", 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131416 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700608+0000", 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020368099999999999 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705556+0000", 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049475760000000004 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705654+0000", 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8444999999999997e-05 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705741+0000", 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6416999999999996e-05 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705863+0000", 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121876 2024-09-13T09:07:35.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.456331245999998, 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:35.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:35.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:35.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.167 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.456023232, 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:35.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:35.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.455935318999998, 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:35.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.185410957999999, 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.185323195000002, 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.185271141999998, 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.185086060000003, 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.185011570999997, 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.184957251, 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.184759165999999, 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.184677903000001, 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.690383187999998, 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.690067550999999, 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.690007221000002, 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.689949315, 2024-09-13T09:07:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.689646717999999, 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7649 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.297081551, 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013081786, 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7649 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853339+0000", 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853345+0000", 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853365+0000", 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9054000000000001e-05 2024-09-13T09:07:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853374+0000", 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858418+0000", 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050442869999999997 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858525+0000", 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010602099999999999 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858992+0000", 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046758299999999999 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.861712+0000", 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027195700000000001 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866315+0000", 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046037439999999999 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866381+0000", 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5121000000000002e-05 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866422+0000", 2024-09-13T09:07:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1331000000000001e-05 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7657 3.10 3:08eae27d:::rbd_data.11242ae5e73a.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13762560~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.282289257, 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018927840000000001, 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7657 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868131+0000", 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870832+0000", 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027013660000000002 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870853+0000", 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.107e-05 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870862+0000", 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7830000000000004e-06 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870910+0000", 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871083+0000", 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017277500000000001 2024-09-13T09:07:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871386+0000", 2024-09-13T09:07:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030378600000000001 2024-09-13T09:07:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.272 INFO:teuthology.orchestra.run.smithi031.stdout:791 punch from 0x3af0630 to 0x3affdd8, (0xf7a8 bytes) 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878951+0000", 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075642649999999997 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886982+0000", 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080316160000000001 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887025+0000", 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3288999999999998e-05 2024-09-13T09:07:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887060+0000", 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4706999999999998e-05 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7660 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.279546168, 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016257230000000001, 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7660 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870874+0000", 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870879+0000", 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870885+0000", 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870888+0000", 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:07:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876842+0000", 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059541780000000001 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876952+0000", 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109577 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.877351+0000", 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039971899999999999 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878995+0000", 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001643179 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887073+0000", 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080780109999999995 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887100+0000", 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7585000000000001e-05 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887133+0000", 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2533000000000002e-05 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.281 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:35.281 INFO:teuthology.orchestra.run.smithi031.stdout:792 read 0x345552d thru 0x345d044 (0x7b18 bytes) 2024-09-13T09:07:35.281 INFO:teuthology.orchestra.run.smithi031.stdout:793 punch from 0xae3221 to 0xae4399, (0x1178 bytes) 2024-09-13T09:07:35.281 INFO:teuthology.orchestra.run.smithi031.stdout:794 write 0xb900d55 thru 0xb90e094 (0xd340 bytes) 2024-09-13T09:07:35.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:35.284 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:35.278+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:35.319 INFO:teuthology.orchestra.run.smithi031.stdout:795 writesame 0x6044ac1 thru 0x6053671 data_size 0x1(0xebb1 bytes) 2024-09-13T09:07:35.384 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:35.382+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:35.429 INFO:teuthology.orchestra.run.smithi031.stdout:796 read 0x7cb0ed0 thru 0x7cb861b (0x774c bytes) 2024-09-13T09:07:35.430 INFO:teuthology.orchestra.run.smithi031.stdout:797 writesame 0x9a9f6f2 thru 0x9aa0b9f data_size 0x1(0x14ae bytes) 2024-09-13T09:07:35.482 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:35.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:35.483 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:35.483 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:35.485 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:35.482+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:35.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:35.568 INFO:teuthology.orchestra.run.smithi031.stdout:798 trunc from 0xd7e01c7 to 0x96935e4 2024-09-13T09:07:35.586 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:35.582+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:35.622 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:35.622 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:35.622 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:35.622 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:35.622 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:35.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:35.651 INFO:teuthology.orchestra.run.smithi031.stdout:799 read 0x8e6aefe thru 0x8e7240a (0x750d bytes) 2024-09-13T09:07:35.652 INFO:teuthology.orchestra.run.smithi031.stdout:800 read 0x5804aaa thru 0x5804fcb (0x522 bytes) 2024-09-13T09:07:35.667 INFO:teuthology.orchestra.run.smithi031.stdout:801 trunc from 0x96935e4 to 0xe3741c 2024-09-13T09:07:35.687 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:35.682+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:35.771 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.771 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.431210276, 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:07:35.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43119350299999998, 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:07:35.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:07:35.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:07:35.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43115619799999999, 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.781 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:35.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43113648399999999, 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:07:35.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:07:35.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.431120159, 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:07:35.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:07:35.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43110429300000003, 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:35.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43108850799999998, 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:35.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:07:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.431069437, 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:07:35.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:07:35.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.808 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43105289000000002, 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:07:35.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43102529099999998, 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:07:35.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.818 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.431008049, 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:35.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:07:35.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43097760600000001, 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:35.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.430958691, 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:07:35.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:07:35.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.833 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43094090299999999, 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.834 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:07:35.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.430915679, 2024-09-13T09:07:35.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:07:35.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43089747499999997, 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.843 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:07:35.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:07:35.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43087715700000001, 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.848 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:35.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.430858609, 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:07:35.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43084133499999999, 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:07:35.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.43082388100000002, 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:35.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:35.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:35.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:35.865 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:35.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:35.867 INFO:teuthology.orchestra.run.smithi031.stdout:802 trunc from 0xe3741c to 0x74cb88e 2024-09-13T09:07:35.868 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:35.864+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:35.869 INFO:teuthology.orchestra.run.smithi031.stdout:803 read 0x5289d20 thru 0x528da9b (0x3d7c bytes) 2024-09-13T09:07:35.870 INFO:teuthology.orchestra.run.smithi031.stdout:804 read 0x4f021ba thru 0x4f08d95 (0x6bdc bytes) 2024-09-13T09:07:35.879 INFO:teuthology.orchestra.run.smithi031.stdout:805 write 0x78e86d5 thru 0x78e9d75 (0x16a1 bytes) 2024-09-13T09:07:35.885 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:35.885 INFO:teuthology.orchestra.run.smithi031.stdout:812 read 0x2d08338 thru 0x2d0ad64 (0x2a2d bytes) 2024-09-13T09:07:35.885 INFO:teuthology.orchestra.run.smithi031.stdout:813 read 0x1b8e7e9 thru 0x1b9b405 (0xcc1d bytes) 2024-09-13T09:07:35.885 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:35.885 INFO:teuthology.orchestra.run.smithi031.stdout:815 read 0x1da3744 thru 0x1da553c (0x1df9 bytes) 2024-09-13T09:07:35.907 INFO:teuthology.orchestra.run.smithi031.stdout:816 write 0xcf3bcac thru 0xcf420db (0x6430 bytes) 2024-09-13T09:07:35.914 INFO:teuthology.orchestra.run.smithi031.stdout:817 writesame 0x6bca157 thru 0x6bd427d data_size 0x25(0xa127 bytes) 2024-09-13T09:07:35.921 INFO:teuthology.orchestra.run.smithi031.stdout:818 writesame 0x435d38 thru 0x43735b data_size 0x1a(0x1624 bytes) 2024-09-13T09:07:35.969 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:35.966+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:36.007 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:36.007 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:36.013 INFO:teuthology.orchestra.run.smithi031.stdout:819 writesame 0x8a3555a thru 0x8a3adcd data_size 0x4(0x5874 bytes) 2024-09-13T09:07:36.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:36.026 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.026 INFO:teuthology.orchestra.run.smithi031.stdout:822 punch from 0x7b45f5e to 0x7b5375e, (0xd800 bytes) 2024-09-13T09:07:36.026 INFO:teuthology.orchestra.run.smithi031.stdout:823 write 0xe61fd6 thru 0xe69148 (0x7173 bytes) 2024-09-13T09:07:36.030 INFO:teuthology.orchestra.run.smithi031.stdout:824 writesame 0xbe6d728 thru 0xbe706d1 data_size 0x1(0x2faa bytes) 2024-09-13T09:07:36.045 INFO:teuthology.orchestra.run.smithi031.stdout:825 writesame 0xb618a1 thru 0xb67322 data_size 0x23(0x5a82 bytes) 2024-09-13T09:07:36.070 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:36.066+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:36.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:36.154 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:36.154 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:36.154 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:36.154 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:36.154 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:36.159 INFO:teuthology.orchestra.run.smithi031.stdout:826 writesame 0xd4dada7 thru 0xd4e1ef8 data_size 0xf(0x7152 bytes) 2024-09-13T09:07:36.162 INFO:teuthology.orchestra.run.smithi031.stdout:827 write 0x41ef244 thru 0x41fa6a9 (0xb466 bytes) 2024-09-13T09:07:36.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:36.171 INFO:teuthology.orchestra.run.smithi031.stdout:828 write 0x6cec8dc thru 0x6cef858 (0x2f7d bytes) 2024-09-13T09:07:36.172 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:36.166+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:36.175 INFO:teuthology.orchestra.run.smithi031.stdout:829 read 0x7dd93ab thru 0x7ddc152 (0x2da8 bytes) 2024-09-13T09:07:36.175 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.185 INFO:teuthology.orchestra.run.smithi031.stdout:833 write 0xd57fc8c thru 0xd583b30 (0x3ea5 bytes) 2024-09-13T09:07:36.188 INFO:teuthology.orchestra.run.smithi031.stdout:834 read 0x175a181 thru 0x1764468 (0xa2e8 bytes) 2024-09-13T09:07:36.207 INFO:teuthology.orchestra.run.smithi031.stdout:835 trunc from 0xd583b31 to 0x1f50946 2024-09-13T09:07:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:836 trunc from 0x1f50946 to 0x37a328f 2024-09-13T09:07:36.250 INFO:teuthology.orchestra.run.smithi031.stdout:837 trunc from 0x37a328f to 0x7143572 2024-09-13T09:07:36.261 INFO:teuthology.orchestra.run.smithi031.stdout:838 writesame 0x414f117 thru 0x415b0ed data_size 0x1(0xbfd7 bytes) 2024-09-13T09:07:36.272 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:36.270+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:36.296 INFO:teuthology.orchestra.run.smithi031.stdout:839 read 0x5180abc thru 0x51816ff (0xc44 bytes) 2024-09-13T09:07:36.296 INFO:teuthology.orchestra.run.smithi031.stdout:840 read 0x208cd89 thru 0x2094768 (0x79e0 bytes) 2024-09-13T09:07:36.296 INFO:teuthology.orchestra.run.smithi031.stdout:842 trunc from 0x7143572 to 0x76921bd 2024-09-13T09:07:36.309 INFO:teuthology.orchestra.run.smithi031.stdout:843 read 0x5716905 thru 0x5724630 (0xdd2c bytes) 2024-09-13T09:07:36.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:07:36.315 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:36.315 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.187447186, 2024-09-13T09:07:36.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:07:36.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:36.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:07:36.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:07:36.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:07:36.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:07:36.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:36.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14831 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0816391730000001, 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071499264000000007, 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14831 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:36.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236541+0000", 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3910000000000001e-06 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236548+0000", 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236553+0000", 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307720+0000", 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071166574999999996 2024-09-13T09:07:36.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307883+0000", 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000163809 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308039+0000", 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155229 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14834 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.080316506, 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070919408000000003, 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14834 2024-09-13T09:07:36.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237864+0000", 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237871+0000", 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237877+0000", 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308614+0000", 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070737169000000003 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308704+0000", 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0357000000000002e-05 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308782+0000", 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7480999999999999e-05 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14835 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:36.336 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0802812810000004, 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071108525000000006, 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14835 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237908+0000", 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1191e-05 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237929+0000", 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1189e-05 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237936+0000", 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:07:36.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308827+0000", 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070891320999999993 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308924+0000", 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6665999999999999e-05 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309006+0000", 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2231000000000006e-05 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14836 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0802363039999996, 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071290567999999999, 2024-09-13T09:07:36.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14836 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:36.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237945+0000", 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6229999999999999e-06 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237953+0000", 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237958+0000", 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0309999999999998e-06 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309041+0000", 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071083576999999995 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309132+0000", 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0500000000000004e-05 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309233+0000", 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101116 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14837 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0802164980000004, 2024-09-13T09:07:36.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071483336999999994, 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14837 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237964+0000", 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237971+0000", 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237975+0000", 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6360000000000002e-06 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309280+0000", 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071304765000000006 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309359+0000", 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9066999999999994e-05 2024-09-13T09:07:36.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309445+0000", 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6255999999999995e-05 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14838 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0801969079999996, 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071691981000000002, 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14838 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237981+0000", 2024-09-13T09:07:36.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237984+0000", 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237990+0000", 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237995+0000", 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309487+0000", 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071492752000000007 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309575+0000", 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7106000000000003e-05 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309674+0000", 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9208000000000003e-05 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14844 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0801794749999996, 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072094264000000005, 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14844 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238002+0000", 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238008+0000", 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.032e-06 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238013+0000", 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309739+0000", 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071726390000000001 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309786+0000", 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6561999999999999e-05 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310093+0000", 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030766000000000002 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14845 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0801613139999997, 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072306747000000005, 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14845 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238019+0000", 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238025+0000", 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4870000000000002e-06 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238029+0000", 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3769999999999998e-06 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310139+0000", 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072110056000000006 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310224+0000", 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4291999999999999e-05 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310324+0000", 2024-09-13T09:07:36.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100405 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14846 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1343488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0801435499999998, 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072637322000000004, 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14846 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:36.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238037+0000", 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238045+0000", 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0960000000000001e-06 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238051+0000", 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310365+0000", 2024-09-13T09:07:36.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072313575000000005 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310432+0000", 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6888000000000001e-05 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310672+0000", 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024050699999999999 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14847 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0801224190000003, 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072846102999999995, 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14847 2024-09-13T09:07:36.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238058+0000", 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5119999999999998e-06 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238063+0000", 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238067+0000", 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4750000000000004e-06 2024-09-13T09:07:36.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310717+0000", 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072649243000000002 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310803+0000", 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6147e-05 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310902+0000", 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9474000000000003e-05 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14848 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0801067880000002, 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072985396999999994, 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14848 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:36.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238074+0000", 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.227e-06 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238080+0000", 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238085+0000", 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1410000000000004e-06 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310939+0000", 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072853862000000005 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310997+0000", 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8248000000000002e-05 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311057+0000", 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0071999999999997e-05 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14849 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0791722930000001, 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072287373000000002, 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14849 2024-09-13T09:07:36.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238089+0000", 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9990826 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239012+0000", 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092334199999999996 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239026+0000", 2024-09-13T09:07:36.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3351e-05 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239034+0000", 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4419999999999998e-06 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311104+0000", 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072069464 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311194+0000", 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9922999999999999e-05 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311294+0000", 2024-09-13T09:07:36.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001002 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14850 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3702784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.079136332, 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072393710999999999, 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14850 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239045+0000", 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4429999999999999e-06 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239053+0000", 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5179999999999998e-06 2024-09-13T09:07:36.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239058+0000", 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311330+0000", 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072271613999999998 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311381+0000", 2024-09-13T09:07:36.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0908000000000001e-05 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311436+0000", 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5513000000000002e-05 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14861 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0789133849999999, 2024-09-13T09:07:36.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072582646000000001, 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14861 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239270+0000", 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-13T09:07:36.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239280+0000", 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0210999999999999e-05 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239285+0000", 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.967e-06 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311481+0000", 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072196249000000004 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311573+0000", 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1588999999999993e-05 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311848+0000", 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027526699999999999 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14862 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0788900750000003, 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072869149999999994, 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14862 2024-09-13T09:07:36.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239288+0000", 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239295+0000", 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239302+0000", 2024-09-13T09:07:36.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.793e-06 2024-09-13T09:07:36.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239305+0000", 2024-09-13T09:07:36.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9380000000000001e-06 2024-09-13T09:07:36.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311878+0000", 2024-09-13T09:07:36.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072573031999999996 2024-09-13T09:07:36.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311923+0000", 2024-09-13T09:07:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4351000000000003e-05 2024-09-13T09:07:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312158+0000", 2024-09-13T09:07:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023512399999999999 2024-09-13T09:07:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.401 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14863 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0788708109999998, 2024-09-13T09:07:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072990372999999997, 2024-09-13T09:07:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14863 2024-09-13T09:07:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:36.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239309+0000", 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939999999999999e-06 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239312+0000", 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239315+0000", 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7630000000000001e-06 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312186+0000", 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072871378000000001 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312243+0000", 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6487000000000003e-05 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312298+0000", 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5506e-05 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14864 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.078811001, 2024-09-13T09:07:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073255022000000003, 2024-09-13T09:07:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14864 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239317+0000", 2024-09-13T09:07:36.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999949 2024-09-13T09:07:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239372+0000", 2024-09-13T09:07:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5457000000000001e-05 2024-09-13T09:07:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239381+0000", 2024-09-13T09:07:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.003e-06 2024-09-13T09:07:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239387+0000", 2024-09-13T09:07:36.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6130000000000003e-06 2024-09-13T09:07:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312327+0000", 2024-09-13T09:07:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072940489999999997 2024-09-13T09:07:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312409+0000", 2024-09-13T09:07:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.153e-05 2024-09-13T09:07:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312623+0000", 2024-09-13T09:07:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213799 2024-09-13T09:07:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14865 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0785404769999998, 2024-09-13T09:07:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073307136999999994, 2024-09-13T09:07:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14865 2024-09-13T09:07:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239390+0000", 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999752 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239643+0000", 2024-09-13T09:07:36.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025296200000000002 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239653+0000", 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7839999999999998e-06 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239660+0000", 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0570000000000004e-06 2024-09-13T09:07:36.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312655+0000", 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072994857999999996 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312731+0000", 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5754999999999996e-05 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312945+0000", 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021457799999999999 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14866 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0785123070000004, 2024-09-13T09:07:36.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073614097000000003, 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14866 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:36.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:36.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239669+0000", 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.858e-06 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239675+0000", 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-13T09:07:36.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239680+0000", 2024-09-13T09:07:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:07:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312975+0000", 2024-09-13T09:07:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073294697000000006 2024-09-13T09:07:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313051+0000", 2024-09-13T09:07:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6297999999999994e-05 2024-09-13T09:07:36.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313280+0000", 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022920800000000001 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:36.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:36.431 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:36.432 INFO:teuthology.orchestra.run.smithi031.stdout:844 write 0xb290a09 thru 0xb29f767 (0xed5f bytes) 2024-09-13T09:07:36.432 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.432 INFO:teuthology.orchestra.run.smithi031.stdout:846 punch from 0x5b676cf to 0x5b68812, (0x1143 bytes) 2024-09-13T09:07:36.432 INFO:teuthology.orchestra.run.smithi031.stdout:847 trunc from 0xb29f768 to 0xb5142f3 2024-09-13T09:07:36.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:36.434 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:36.434 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:36.435 INFO:teuthology.orchestra.run.smithi031.stdout:848 read 0xe793be thru 0xe8304f (0x9c92 bytes) 2024-09-13T09:07:36.435 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.435 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.435 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:36.436 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:36.436 INFO:teuthology.orchestra.run.smithi031.stdout:851 write 0xbfd9ee9 thru 0xbfe62f8 (0xc410 bytes) 2024-09-13T09:07:36.436 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.436 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.437 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.437 INFO:teuthology.orchestra.run.smithi031.stdout:855 read 0x7f46aa3 thru 0x7f56362 (0xf8c0 bytes) 2024-09-13T09:07:36.437 INFO:teuthology.orchestra.run.smithi031.stdout:857 write 0x81a4b43 thru 0x81af7ac (0xac6a bytes) 2024-09-13T09:07:36.437 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:36.437 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:36.438 INFO:teuthology.orchestra.run.smithi031.stdout:858 write 0x1ed473a thru 0x1ee0992 (0xc259 bytes) 2024-09-13T09:07:36.438 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.438 INFO:teuthology.orchestra.run.smithi031.stdout:862 writesame 0xb64a50a thru 0xb654aaf data_size 0x5b(0xa5a6 bytes) 2024-09-13T09:07:36.438 INFO:teuthology.orchestra.run.smithi031.stdout:863 read 0x6366cc2 thru 0x63763b8 (0xf6f7 bytes) 2024-09-13T09:07:36.439 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:36.439 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:36.439 INFO:teuthology.orchestra.run.smithi031.stdout:866 trunc from 0xbfe62f9 to 0x10347d8 2024-09-13T09:07:36.440 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:36.428+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:36.440 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:36.441 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:36.443 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.458 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:36.458 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:36.466 INFO:teuthology.orchestra.run.smithi031.stdout:868 write 0xa075644 thru 0xa07628b (0xc48 bytes) 2024-09-13T09:07:36.471 INFO:teuthology.orchestra.run.smithi031.stdout:870 punch from 0x4c2d092 to 0x4c3ab26, (0xda94 bytes) 2024-09-13T09:07:36.472 INFO:teuthology.orchestra.run.smithi031.stdout:871 punch from 0x43d12b1 to 0x43d502d, (0x3d7c bytes) 2024-09-13T09:07:36.472 INFO:teuthology.orchestra.run.smithi031.stdout:873 write 0x3431fe6 thru 0x3434ad5 (0x2af0 bytes) 2024-09-13T09:07:36.478 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:36.479 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:36.480 INFO:teuthology.orchestra.run.smithi031.stdout:874 trunc from 0xa07628c to 0x9ef3441 2024-09-13T09:07:36.504 INFO:teuthology.orchestra.run.smithi031.stdout:875 trunc from 0x9ef3441 to 0x2729729 2024-09-13T09:07:36.533 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:36.530+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:36.542 INFO:teuthology.orchestra.run.smithi031.stdout:877 write 0xaf01c82 thru 0xaf0b008 (0x9387 bytes) 2024-09-13T09:07:36.564 INFO:teuthology.orchestra.run.smithi031.stdout:878 write 0xd628ae1 thru 0xd631f14 (0x9434 bytes) 2024-09-13T09:07:36.574 INFO:teuthology.orchestra.run.smithi031.stdout:879 read 0xb6f01f thru 0xb730e5 (0x40c7 bytes) 2024-09-13T09:07:36.575 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:36.575 INFO:teuthology.orchestra.run.smithi031.stdout:884 read 0x4d5ce69 thru 0x4d5e99e (0x1b36 bytes) 2024-09-13T09:07:36.575 INFO:teuthology.orchestra.run.smithi031.stdout:885 read 0x378979c thru 0x379537f (0xbbe4 bytes) 2024-09-13T09:07:36.575 INFO:teuthology.orchestra.run.smithi031.stdout:888 read 0xb2d2ea3 thru 0xb2dfb18 (0xcc76 bytes) 2024-09-13T09:07:36.576 INFO:teuthology.orchestra.run.smithi031.stdout:889 writesame 0x2bc4d4c thru 0x2bc92fe data_size 0x1(0x45b3 bytes) 2024-09-13T09:07:36.586 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:36.586 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:36.594 INFO:teuthology.orchestra.run.smithi031.stdout:890 write 0x339e738 thru 0x33a8a58 (0xa321 bytes) 2024-09-13T09:07:36.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:36.606 INFO:teuthology.orchestra.run.smithi031.stdout:891 write 0x2ed1172 thru 0x2ed8d94 (0x7c23 bytes) 2024-09-13T09:07:36.612 INFO:teuthology.orchestra.run.smithi031.stdout:892 writesame 0x943cb29 thru 0x944c3f3 data_size 0x1(0xf8cb bytes) 2024-09-13T09:07:36.634 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:36.628+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:36.696 INFO:teuthology.orchestra.run.smithi031.stdout:893 punch from 0x78b6d31 to 0x78b7a15, (0xce4 bytes) 2024-09-13T09:07:36.707 INFO:teuthology.orchestra.run.smithi031.stdout:894 write 0xd94b96e thru 0xd957c0e (0xc2a1 bytes) 2024-09-13T09:07:36.716 INFO:teuthology.orchestra.run.smithi031.stdout:895 write 0x7e09bb8 thru 0x7e18354 (0xe79d bytes) 2024-09-13T09:07:36.723 INFO:teuthology.orchestra.run.smithi031.stdout:896 clone 10 order 18 su 32768 sc 14 2024-09-13T09:07:36.735 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:36.730+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:36.766 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:36.767 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:36.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:36.835 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:36.832+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:36.936 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:36.934+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:36.949 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:36.950 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:36.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:37.037 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:37.032+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:37.131 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:37.131 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:37.138 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:37.134+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:37.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:37.240 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:37.236+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:37.304 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:07:37.321 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:37.321 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:37.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:37.371 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:37.368+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:37.417 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone9 from 0xd957c0f (overlap 0xe3741c) to 0x0 2024-09-13T09:07:37.442 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:37.438+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:37.468 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:37.468 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:37.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:37.541 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:37.536+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:37.610 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-13T09:07:37.642 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:37.636+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:37.656 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:37.656 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:37.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:37.676 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:37.676 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:07:37.676 INFO:tasks.thrashosds.thrasher:Reweighting osd 5 to 0.8765021834575379 2024-09-13T09:07:37.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 0.8765021834575379 2024-09-13T09:07:37.743 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:37.738+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:37.799 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:37.799 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:37.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:37.844 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:37.840+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:37.945 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:37.942+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:37.985 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:37.985 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:37.999 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:38.046 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:38.040+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:38.085 INFO:teuthology.orchestra.run.smithi031.stdout:897 read 0x4259ce8 thru 0x426896b (0xec84 bytes) 2024-09-13T09:07:38.086 INFO:teuthology.orchestra.run.smithi031.stdout:899 punch from 0x7562ce7 to 0x7570bd9, (0xdef2 bytes) 2024-09-13T09:07:38.091 INFO:teuthology.orchestra.run.smithi031.stdout:902 trunc from 0xd957c0f to 0xd989428 2024-09-13T09:07:38.101 INFO:teuthology.orchestra.run.smithi031.stdout:903 write 0x80ee9ff thru 0x80fc2ca (0xd8cc bytes) 2024-09-13T09:07:38.116 INFO:teuthology.orchestra.run.smithi031.stdout:904 write 0x4c79e30 thru 0x4c8419a (0xa36b bytes) 2024-09-13T09:07:38.124 INFO:teuthology.orchestra.run.smithi031.stdout:905 punch from 0xba3cd7 to 0xbb3063, (0xf38c bytes) 2024-09-13T09:07:38.124 INFO:teuthology.orchestra.run.smithi031.stdout:908 write 0x80f7e4a thru 0x80fa336 (0x24ed bytes) 2024-09-13T09:07:38.130 INFO:teuthology.orchestra.run.smithi031.stdout:909 flatten 2024-09-13T09:07:38.147 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:38.142+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:38.185 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:38.185 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:38.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:38.288 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:38.286+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:38.304 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 0.876502 (e062) 2024-09-13T09:07:38.349 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:38.346+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:38.404 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:38.405 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:38.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:38.450 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:38.444+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:38.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:38.550 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:38.546+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:38.630 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:38.630 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:38.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:38.675 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:38.675 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:38.675 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:38.672+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:38.693 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:38.693 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:38.711 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:38.727 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:38.727 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:38.747 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:38.747 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:38.751 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:38.748+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:38.764 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:38.764 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:38.766 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:38.767 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:38.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:38.788 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:38.788 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:38.809 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:38.809 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:38.874 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:38.872+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:38.913 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:38.913 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:38.913 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:38.913 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:38.913 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:38.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:38.953 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:38.950+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:39.054 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:39.048+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:39.102 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:39.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:15159 3.c 3:33b101a6:::rbd_data.11242e26750a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15794176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.8051077949999996, 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052673589999999999, 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 15159 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294458+0000", 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302503+0000", 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080450420000000005 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302534+0000", 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1352000000000001e-05 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302550+0000", 2024-09-13T09:07:39.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5511000000000001e-05 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302617+0000", 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7722e-05 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302628+0000", 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0997e-05 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304649+0000", 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020203460000000001 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304734+0000", 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4839999999999994e-05 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.305520+0000", 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078672300000000002 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.343824+0000", 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038304108000000003 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347049+0000", 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032249399999999999 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347102+0000", 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2163000000000002e-05 2024-09-13T09:07:39.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347133+0000", 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1674999999999997e-05 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20975 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4112384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7473610009999998, 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052586704999999997, 2024-09-13T09:07:39.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20975 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352209+0000", 2024-09-13T09:07:39.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.886e-06 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352216+0000", 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4159999999999998e-06 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352222+0000", 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404606+0000", 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052384209000000001 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404695+0000", 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8455000000000006e-05 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404793+0000", 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8443999999999995e-05 2024-09-13T09:07:39.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20976 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7473402359999999, 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052710422999999999, 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20976 2024-09-13T09:07:39.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352230+0000", 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4829999999999999e-06 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352238+0000", 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.14e-06 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352245+0000", 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3130000000000004e-06 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404836+0000", 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052590779999999997 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404881+0000", 2024-09-13T09:07:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5034000000000003e-05 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404938+0000", 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7120999999999997e-05 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20977 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4227072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7473176170000002, 2024-09-13T09:07:39.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052908033, 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20977 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:39.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352249+0000", 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352259+0000", 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9520000000000006e-06 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352268+0000", 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5550000000000003e-06 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352274+0000", 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404981+0000", 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052706483999999998 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405058+0000", 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7282999999999999e-05 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405158+0000", 2024-09-13T09:07:39.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9699999999999998e-05 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20978 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.747261054, 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053008803, 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20978 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:39.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352305+0000", 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352312+0000", 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5339999999999996e-06 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352325+0000", 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2952000000000001e-05 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352333+0000", 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0180000000000001e-06 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405203+0000", 2024-09-13T09:07:39.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052870013 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405257+0000", 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4135000000000003e-05 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405315+0000", 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8471999999999999e-05 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20979 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4308992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7472280109999998, 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053200718000000001, 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20979 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352342+0000", 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7610000000000001e-06 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352349+0000", 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:07:39.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352355+0000", 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405359+0000", 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053004355000000003 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405440+0000", 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0927999999999995e-05 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405540+0000", 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010015599999999999 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20980 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7472070789999998, 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053319995000000002, 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20980 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352362+0000", 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5959999999999998e-06 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352367+0000", 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4560000000000002e-06 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352370+0000", 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0860000000000002e-06 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405580+0000", 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053209583999999997 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405619+0000", 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9617999999999999e-05 2024-09-13T09:07:39.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405680+0000", 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1259000000000004e-05 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20981 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4423680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7471949659999999, 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053531664999999999, 2024-09-13T09:07:39.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20981 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352372+0000", 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352374+0000", 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5519999999999999e-06 2024-09-13T09:07:39.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352377+0000", 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.258e-06 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352388+0000", 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1375e-05 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405720+0000", 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053331770000000001 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405800+0000", 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0050999999999996e-05 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405904+0000", 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103879 2024-09-13T09:07:39.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20982 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4456448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.747176192, 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053719282, 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20982 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352393+0000", 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.615e-06 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352396+0000", 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6090000000000001e-06 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352399+0000", 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405931+0000", 2024-09-13T09:07:39.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053532140999999998 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406011+0000", 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0226000000000001e-05 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406111+0000", 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9087999999999997e-05 2024-09-13T09:07:39.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20983 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4505600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.746941815, 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053724453999999998, 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20983 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352401+0000", 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997754 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352631+0000", 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000230134 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352644+0000", 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2308e-05 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352651+0000", 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2030000000000003e-06 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406162+0000", 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053510765000000002 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406241+0000", 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9764000000000004e-05 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406350+0000", 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010872199999999999 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20992 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4931584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7437174940000002, 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052559458000000003, 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20992 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355849+0000", 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355852+0000", 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2109999999999998e-06 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355859+0000", 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355864+0000", 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.028e-06 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408296+0000", 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052431658999999999 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408350+0000", 2024-09-13T09:07:39.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4253999999999999e-05 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408409+0000", 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9247999999999999e-05 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20993 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5013504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:39.156 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7436777879999998, 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052710119999999999, 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20993 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355889+0000", 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355893+0000", 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2509999999999996e-06 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355903+0000", 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355909+0000", 2024-09-13T09:07:39.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0190000000000003e-06 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408441+0000", 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052532395000000003 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408515+0000", 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4090000000000004e-05 2024-09-13T09:07:39.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408600+0000", 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4716999999999995e-05 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20994 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5046272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.743653412, 2024-09-13T09:07:39.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052842215999999997, 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20994 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355913+0000", 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355916+0000", 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355923+0000", 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355931+0000", 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5280000000000003e-06 2024-09-13T09:07:39.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408630+0000", 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052698997999999997 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408698+0000", 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7966999999999995e-05 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408756+0000", 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8233000000000003e-05 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20995 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5095424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.743630365, 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052955222000000003, 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20995 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355940+0000", 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1669999999999999e-06 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355946+0000", 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355951+0000", 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9309999999999999e-06 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408783+0000", 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052831751000000003 2024-09-13T09:07:39.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408838+0000", 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5031000000000002e-05 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408892+0000", 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4134e-05 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20996 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5128192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7436113710000001, 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053075004000000002, 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20996 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355959+0000", 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355965+0000", 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-13T09:07:39.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355972+0000", 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408922+0000", 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052950174000000003 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408973+0000", 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0837000000000001e-05 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409031+0000", 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8455000000000002e-05 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20997 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5210112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:39.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.743591323, 2024-09-13T09:07:39.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053241038999999997, 2024-09-13T09:07:39.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20997 2024-09-13T09:07:39.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355979+0000", 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0170000000000001e-06 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355985+0000", 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.122e-06 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355990+0000", 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409061+0000", 2024-09-13T09:07:39.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053071209000000001 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409127+0000", 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5704999999999995e-05 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409217+0000", 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0492e-05 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20998 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5242880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.7435729159999998, 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053395182999999999, 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20998 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355994+0000", 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355997+0000", 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7470000000000001e-06 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356003+0000", 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356009+0000", 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1990000000000003e-06 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409244+0000", 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053234721999999998 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409321+0000", 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6998999999999999e-05 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409390+0000", 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8943999999999996e-05 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20999 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5292032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.74355386, 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053549106999999999, 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20999 2024-09-13T09:07:39.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356013+0000", 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356016+0000", 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.016e-06 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356023+0000", 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3810000000000004e-06 2024-09-13T09:07:39.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356038+0000", 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5305999999999999e-05 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409408+0000", 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053369650999999997 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409472+0000", 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4587999999999999e-05 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409563+0000", 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0566e-05 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:21000 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.743522886, 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053705747999999998, 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21000 2024-09-13T09:07:39.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356044+0000", 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356048+0000", 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.985e-06 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356056+0000", 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356061+0000", 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409590+0000", 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053528986000000001 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409663+0000", 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3342000000000004e-05 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409750+0000", 2024-09-13T09:07:39.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7349999999999995e-05 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:21001 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.743499532, 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053808105000000002, 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21001 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356067+0000", 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356080+0000", 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2228999999999999e-05 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356088+0000", 2024-09-13T09:07:39.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356093+0000", 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409777+0000", 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053683869000000002 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409824+0000", 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6814000000000003e-05 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409876+0000", 2024-09-13T09:07:39.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2405999999999999e-05 2024-09-13T09:07:39.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.201 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:39.202 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:39.203 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:39.200+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:39.304 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:39.298+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:39.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:07:39.381 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:39.381 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:39.381 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:39.381 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:39.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:39.443 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:39.440+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:39.505 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:39.502+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:39.523 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:39.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:39.523 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:39.523 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:39.523 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:39.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:39.606 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:39.602+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.009278215999998, 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840076, 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 34 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694022+0000", 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694033+0000", 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.116e-05 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694051+0000", 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694065+0000", 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3716e-05 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700273+0000", 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062082320000000002 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700405+0000", 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131416 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700608+0000", 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020368099999999999 2024-09-13T09:07:39.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705556+0000", 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049475760000000004 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705654+0000", 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8444999999999997e-05 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705741+0000", 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6416999999999996e-05 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705863+0000", 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121876 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.009210379000002, 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:39.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.008902364999997, 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:39.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.008814452000003, 2024-09-13T09:07:39.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:39.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.738290091000003, 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:39.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:39.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.738202328, 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.739 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:39.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:39.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:39.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.738150275000002, 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:39.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.737965193000001, 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:39.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:39.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:39.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.737890704000002, 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.737836383999998, 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.737638298999997, 2024-09-13T09:07:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:39.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.737557035999998, 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.243262321000003, 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.242946684000003, 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.242886353999999, 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.242828447999997, 2024-09-13T09:07:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.242525851000003, 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7649 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.849960683999999, 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013081786, 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7649 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853339+0000", 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853345+0000", 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853365+0000", 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9054000000000001e-05 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853374+0000", 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858418+0000", 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050442869999999997 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858525+0000", 2024-09-13T09:07:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010602099999999999 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858992+0000", 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046758299999999999 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.861712+0000", 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027195700000000001 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866315+0000", 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046037439999999999 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866381+0000", 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5121000000000002e-05 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866422+0000", 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1331000000000001e-05 2024-09-13T09:07:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7657 3.10 3:08eae27d:::rbd_data.11242ae5e73a.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13762560~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.83516839, 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018927840000000001, 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7657 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868131+0000", 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870832+0000", 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027013660000000002 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870853+0000", 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.107e-05 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870862+0000", 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7830000000000004e-06 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870910+0000", 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871083+0000", 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017277500000000001 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871386+0000", 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030378600000000001 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878951+0000", 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075642649999999997 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886982+0000", 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080316160000000001 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887025+0000", 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3288999999999998e-05 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887060+0000", 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4706999999999998e-05 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7660 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.832425301000001, 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016257230000000001, 2024-09-13T09:07:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7660 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870874+0000", 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870879+0000", 2024-09-13T09:07:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870885+0000", 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870888+0000", 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876842+0000", 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059541780000000001 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876952+0000", 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109577 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.877351+0000", 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039971899999999999 2024-09-13T09:07:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878995+0000", 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001643179 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887073+0000", 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080780109999999995 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887100+0000", 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7585000000000001e-05 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887133+0000", 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2533000000000002e-05 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:39.832 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:39.833 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:39.704+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:39.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:39.875 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:39.872+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:39.936 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:39.932+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:40.007 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:40.007 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:40.008 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:40.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:40.048 INFO:teuthology.orchestra.run.smithi031.stdout:913 write 0x55be765 thru 0x55cd616 (0xeeb2 bytes) 2024-09-13T09:07:40.048 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:40.044+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:40.056 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:40.079 INFO:teuthology.orchestra.run.smithi031.stdout:917 trunc from 0xd989428 to 0x333d1c5 2024-09-13T09:07:40.105 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:40.122 INFO:teuthology.orchestra.run.smithi031.stdout:922 write 0x5e28836 thru 0x5e34d99 (0xc564 bytes) 2024-09-13T09:07:40.131 INFO:teuthology.orchestra.run.smithi031.stdout:924 trunc from 0x5e34d9a to 0x790d295 2024-09-13T09:07:40.138 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:40.134+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:40.158 INFO:teuthology.orchestra.run.smithi031.stdout:925 write 0xa30666d thru 0xa30f529 (0x8ebd bytes) 2024-09-13T09:07:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:926 read 0x7472f2b thru 0x747bc4b (0x8d21 bytes) 2024-09-13T09:07:40.186 INFO:teuthology.orchestra.run.smithi031.stdout:928 trunc from 0xa30f52a to 0x21b3ec3 2024-09-13T09:07:40.188 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:40.188 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:40.188 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:40.188 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:40.188 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:40.206 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:40.213 INFO:teuthology.orchestra.run.smithi031.stdout:929 clone 11 order 24 su 1048576 sc 14 2024-09-13T09:07:40.258 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:40.256+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:40.323 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:07:40.340 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:40.338+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:40.346 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:40.346 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:40.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:40.346 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0072954080000001, 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:07:40.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:07:40.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0072786349999996, 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:07:40.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:07:40.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0072413300000003, 2024-09-13T09:07:40.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:07:40.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0072216159999998, 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:40.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:07:40.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.007205291, 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:07:40.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.007189425, 2024-09-13T09:07:40.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:07:40.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:07:40.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.375 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.375 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.375 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:40.375 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0071736400000004, 2024-09-13T09:07:40.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:07:40.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:07:40.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:07:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:07:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0071545689999999, 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:07:40.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:07:40.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:07:40.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0071380220000004, 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:07:40.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0071104230000003, 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:07:40.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:07:40.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:07:40.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0070931810000001, 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:40.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0070627380000001, 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:07:40.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:07:40.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0070438230000001, 2024-09-13T09:07:40.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:40.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:07:40.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.007026035, 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:07:40.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.413 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0070008110000002, 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:07:40.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:07:40.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0069826070000003, 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:07:40.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:40.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:07:40.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0069622889999996, 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:07:40.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:07:40.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0069437409999997, 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:07:40.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.432 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0069264670000004, 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:07:40.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0069090129999996, 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:07:40.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.442 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:40.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:40.444 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:40.438+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:40.514 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-13T09:07:40.542 INFO:teuthology.orchestra.run.smithi031.stdout:930 writesame 0x712d252 thru 0x713c409 data_size 0xb6(0xf1b8 bytes) 2024-09-13T09:07:40.543 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:40.538+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:40.574 INFO:teuthology.orchestra.run.smithi031.stdout:931 writesame 0xe82a8ea thru 0xe830208 data_size 0x41(0x591f bytes) 2024-09-13T09:07:40.575 INFO:teuthology.orchestra.run.smithi031.stdout:932 read 0xc4ecb22 thru 0xc4f053f (0x3a1e bytes) 2024-09-13T09:07:40.575 INFO:teuthology.orchestra.run.smithi031.stdout:933 read 0x378444a thru 0x378f798 (0xb34f bytes) 2024-09-13T09:07:40.575 INFO:teuthology.orchestra.run.smithi031.stdout:934 write 0x6964cc3 thru 0x6973e65 (0xf1a3 bytes) 2024-09-13T09:07:40.581 INFO:teuthology.orchestra.run.smithi031.stdout:935 writesame 0x8340f98 thru 0x834ea27 data_size 0x34(0xda90 bytes) 2024-09-13T09:07:40.614 INFO:teuthology.orchestra.run.smithi031.stdout:937 trunc from 0xe830209 to 0x22fc2ea 2024-09-13T09:07:40.633 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:40.633 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:40.633 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:40.644 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:40.638+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:40.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:40.662 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:40.662 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:40.692 INFO:teuthology.orchestra.run.smithi031.stdout:940 write 0xc9cbc88 thru 0xc9da669 (0xe9e2 bytes) 2024-09-13T09:07:40.695 INFO:teuthology.orchestra.run.smithi031.stdout:941 flatten 2024-09-13T09:07:40.704 INFO:teuthology.orchestra.run.smithi031.stdout:943 write 0x10af696 thru 0x10b11d5 (0x1b40 bytes) 2024-09-13T09:07:40.706 INFO:teuthology.orchestra.run.smithi031.stdout:946 read 0x7a5b3c1 thru 0x7a61995 (0x65d5 bytes) 2024-09-13T09:07:40.707 INFO:teuthology.orchestra.run.smithi031.stdout:948 write 0x387024c thru 0x387531b (0x50d0 bytes) 2024-09-13T09:07:40.711 INFO:teuthology.orchestra.run.smithi031.stdout:949 writesame 0xab4759c thru 0xab528b8 data_size 0x1(0xb31d bytes) 2024-09-13T09:07:40.744 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:40.742+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:40.749 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:40.750 INFO:teuthology.orchestra.run.smithi031.stdout:952 writesame 0x49f4cb5 thru 0x49f9600 data_size 0x2(0x494c bytes) 2024-09-13T09:07:40.760 INFO:teuthology.orchestra.run.smithi031.stdout:953 punch from 0x7f6570 to 0x80210f, (0xbb9f bytes) 2024-09-13T09:07:40.760 INFO:teuthology.orchestra.run.smithi031.stdout:954 read 0xa1c40a3 thru 0xa1c8832 (0x4790 bytes) 2024-09-13T09:07:40.761 INFO:teuthology.orchestra.run.smithi031.stdout:955 write 0x131d8aa thru 0x131f6bd (0x1e14 bytes) 2024-09-13T09:07:40.765 INFO:teuthology.orchestra.run.smithi031.stdout:956 writesame 0xb8040c9 thru 0xb813bbf data_size 0x1(0xfaf7 bytes) 2024-09-13T09:07:40.795 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:40.795 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:40.795 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:40.815 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:40.828 INFO:teuthology.orchestra.run.smithi031.stdout:957 write 0x34f07b3 thru 0x34fc6cd (0xbf1b bytes) 2024-09-13T09:07:40.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:40.833 INFO:teuthology.orchestra.run.smithi031.stdout:959 write 0xabdd94c thru 0xabeaa09 (0xd0be bytes) 2024-09-13T09:07:40.842 INFO:teuthology.orchestra.run.smithi031.stdout:960 trunc from 0xc9da66a to 0x8e9876a 2024-09-13T09:07:40.845 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:40.842+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:40.861 INFO:teuthology.orchestra.run.smithi031.stdout:961 read 0x2a479d thru 0x2acec1 (0x8725 bytes) 2024-09-13T09:07:40.862 INFO:teuthology.orchestra.run.smithi031.stdout:962 read 0x3ead666 thru 0x3eadc4e (0x5e9 bytes) 2024-09-13T09:07:40.862 INFO:teuthology.orchestra.run.smithi031.stdout:963 punch from 0x825eca4 to 0x8267a41, (0x8d9d bytes) 2024-09-13T09:07:40.862 INFO:teuthology.orchestra.run.smithi031.stdout:964 read 0x2668d0f thru 0x2678833 (0xfb25 bytes) 2024-09-13T09:07:40.873 INFO:teuthology.orchestra.run.smithi031.stdout:966 trunc from 0x8e9876a to 0x2d559b3 2024-09-13T09:07:40.892 INFO:teuthology.orchestra.run.smithi031.stdout:967 trunc from 0x2d559b3 to 0x52899e 2024-09-13T09:07:40.938 INFO:teuthology.orchestra.run.smithi031.stdout:968 read 0x4cbba4 thru 0x4d8a27 (0xce84 bytes) 2024-09-13T09:07:40.946 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:40.944+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:40.949 INFO:teuthology.orchestra.run.smithi031.stdout:969 write 0x19de00c thru 0x19e792a (0x991f bytes) 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.830457535999997, 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:07:40.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:07:40.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:07:40.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:07:40.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.969 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14831 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7246495230000001, 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071499264000000007, 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14831 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236541+0000", 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3910000000000001e-06 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236548+0000", 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236553+0000", 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307720+0000", 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071166574999999996 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307883+0000", 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000163809 2024-09-13T09:07:40.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308039+0000", 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155229 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14834 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7233268559999999, 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070919408000000003, 2024-09-13T09:07:40.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14834 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237864+0000", 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237871+0000", 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237877+0000", 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308614+0000", 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070737169000000003 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308704+0000", 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0357000000000002e-05 2024-09-13T09:07:40.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308782+0000", 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7480999999999999e-05 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14835 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7232916310000004, 2024-09-13T09:07:40.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071108525000000006, 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14835 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237908+0000", 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1191e-05 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237929+0000", 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1189e-05 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237936+0000", 2024-09-13T09:07:40.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308827+0000", 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070891320999999993 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308924+0000", 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6665999999999999e-05 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309006+0000", 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2231000000000006e-05 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14836 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7232466540000004, 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071290567999999999, 2024-09-13T09:07:40.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14836 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237945+0000", 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6229999999999999e-06 2024-09-13T09:07:40.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237953+0000", 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237958+0000", 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0309999999999998e-06 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309041+0000", 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071083576999999995 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309132+0000", 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0500000000000004e-05 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309233+0000", 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101116 2024-09-13T09:07:40.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14837 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7232268479999995, 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071483336999999994, 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14837 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:40.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237964+0000", 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237971+0000", 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237975+0000", 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6360000000000002e-06 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309280+0000", 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071304765000000006 2024-09-13T09:07:40.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309359+0000", 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9066999999999994e-05 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309445+0000", 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6255999999999995e-05 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14838 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7232072580000004, 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071691981000000002, 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14838 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.994 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:40.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237981+0000", 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237984+0000", 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237990+0000", 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:07:40.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237995+0000", 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309487+0000", 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071492752000000007 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:40.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309575+0000", 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7106000000000003e-05 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309674+0000", 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9208000000000003e-05 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14844 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:40.998 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231898250000004, 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072094264000000005, 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14844 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:40.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238002+0000", 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238008+0000", 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.032e-06 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238013+0000", 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:07:41.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309739+0000", 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071726390000000001 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309786+0000", 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6561999999999999e-05 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310093+0000", 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030766000000000002 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14845 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231716640000005, 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072306747000000005, 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14845 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:41.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238019+0000", 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238025+0000", 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4870000000000002e-06 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238029+0000", 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3769999999999998e-06 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310139+0000", 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072110056000000006 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310224+0000", 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4291999999999999e-05 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310324+0000", 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100405 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14846 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1343488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231538999999998, 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072637322000000004, 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14846 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:41.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238037+0000", 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238045+0000", 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0960000000000001e-06 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238051+0000", 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310365+0000", 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072313575000000005 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310432+0000", 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6888000000000001e-05 2024-09-13T09:07:41.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310672+0000", 2024-09-13T09:07:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024050699999999999 2024-09-13T09:07:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14847 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231327689999993, 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072846102999999995, 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14847 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238058+0000", 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5119999999999998e-06 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238063+0000", 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238067+0000", 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4750000000000004e-06 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310717+0000", 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072649243000000002 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310803+0000", 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6147e-05 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310902+0000", 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9474000000000003e-05 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14848 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231171379999992, 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072985396999999994, 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14848 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238074+0000", 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.227e-06 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238080+0000", 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238085+0000", 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1410000000000004e-06 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310939+0000", 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072853862000000005 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310997+0000", 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8248000000000002e-05 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311057+0000", 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0071999999999997e-05 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14849 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.722182643, 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072287373000000002, 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14849 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:41.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238089+0000", 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9990826 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239012+0000", 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092334199999999996 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239026+0000", 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3351e-05 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239034+0000", 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4419999999999998e-06 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311104+0000", 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072069464 2024-09-13T09:07:41.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311194+0000", 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9922999999999999e-05 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311294+0000", 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001002 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14850 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3702784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.722146682, 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072393710999999999, 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14850 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239045+0000", 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4429999999999999e-06 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239053+0000", 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5179999999999998e-06 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239058+0000", 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311330+0000", 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072271613999999998 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311381+0000", 2024-09-13T09:07:41.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0908000000000001e-05 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311436+0000", 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5513000000000002e-05 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14861 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:41.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7219237350000007, 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072582646000000001, 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14861 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239270+0000", 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239280+0000", 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0210999999999999e-05 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239285+0000", 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.967e-06 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311481+0000", 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072196249000000004 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311573+0000", 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1588999999999993e-05 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311848+0000", 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027526699999999999 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14862 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7219004249999994, 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072869149999999994, 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14862 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239288+0000", 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239295+0000", 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239302+0000", 2024-09-13T09:07:41.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.793e-06 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239305+0000", 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9380000000000001e-06 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311878+0000", 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072573031999999996 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311923+0000", 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4351000000000003e-05 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312158+0000", 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023512399999999999 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14863 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.040 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7218811610000007, 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072990372999999997, 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14863 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:41.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239309+0000", 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939999999999999e-06 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239312+0000", 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239315+0000", 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7630000000000001e-06 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312186+0000", 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072871378000000001 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312243+0000", 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6487000000000003e-05 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312298+0000", 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5506e-05 2024-09-13T09:07:41.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14864 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7218213509999991, 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073255022000000003, 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14864 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239317+0000", 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999949 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239372+0000", 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5457000000000001e-05 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239381+0000", 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.003e-06 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239387+0000", 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6130000000000003e-06 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312327+0000", 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072940489999999997 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312409+0000", 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.153e-05 2024-09-13T09:07:41.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312623+0000", 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213799 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14865 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7215508269999997, 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073307136999999994, 2024-09-13T09:07:41.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14865 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239390+0000", 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999752 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239643+0000", 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025296200000000002 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239653+0000", 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7839999999999998e-06 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239660+0000", 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0570000000000004e-06 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312655+0000", 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072994857999999996 2024-09-13T09:07:41.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312731+0000", 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5754999999999996e-05 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312945+0000", 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021457799999999999 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14866 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7215226569999995, 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073614097000000003, 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14866 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:41.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239669+0000", 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.858e-06 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239675+0000", 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239680+0000", 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312975+0000", 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073294697000000006 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313051+0000", 2024-09-13T09:07:41.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6297999999999994e-05 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313280+0000", 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022920800000000001 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:41.058 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:41.059 INFO:teuthology.orchestra.run.smithi031.stdout:971 write 0x6ccfb79 thru 0x6cdb17c (0xb604 bytes) 2024-09-13T09:07:41.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:41.061 INFO:teuthology.orchestra.run.smithi031.stdout:972 trunc from 0x6cdb17d to 0x9043fe6 2024-09-13T09:07:41.061 INFO:teuthology.orchestra.run.smithi031.stdout:973 punch from 0x44bf804 to 0x44c0105, (0x901 bytes) 2024-09-13T09:07:41.061 INFO:teuthology.orchestra.run.smithi031.stdout:974 punch from 0x3bbfa0d to 0x3bc98c9, (0x9ebc bytes) 2024-09-13T09:07:41.061 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.061 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:41.062 INFO:teuthology.orchestra.run.smithi031.stdout:976 trunc from 0x9043fe6 to 0x1e9b97d 2024-09-13T09:07:41.062 INFO:teuthology.orchestra.run.smithi031.stdout:977 read 0x44b829 thru 0x44e4e7 (0x2cbf bytes) 2024-09-13T09:07:41.062 INFO:teuthology.orchestra.run.smithi031.stdout:979 read 0x51bf8d thru 0x5244fb (0x856f bytes) 2024-09-13T09:07:41.063 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:41.063 INFO:teuthology.orchestra.run.smithi031.stdout:980 write 0xb06ea20 thru 0xb07ceb9 (0xe49a bytes) 2024-09-13T09:07:41.063 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:41.064 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:41.064 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:983 write 0x4f3bd50 thru 0x4f49ebc (0xe16d bytes) 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:987 read 0x1f8f869 thru 0x1f98e7b (0x9613 bytes) 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:988 read 0x63476b7 thru 0x6350de0 (0x972a bytes) 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:989 punch from 0x6497d0 to 0x64de79, (0x46a9 bytes) 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:991 punch from 0x797e4f9 to 0x798613c, (0x7c43 bytes) 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:992 write 0xb9994ec thru 0xb9a65f0 (0xd105 bytes) 2024-09-13T09:07:41.065 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:41.066 INFO:teuthology.orchestra.run.smithi031.stdout:995 write 0x5da84a8 thru 0x5da9b68 (0x16c1 bytes) 2024-09-13T09:07:41.066 INFO:teuthology.orchestra.run.smithi031.stdout:997 read 0x438cfa3 thru 0x43941a6 (0x7204 bytes) 2024-09-13T09:07:41.066 INFO:teuthology.orchestra.run.smithi031.stdout:998 trunc from 0xb9a65f1 to 0xa3b9fac 2024-09-13T09:07:41.067 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:41.054+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:41.076 INFO:teuthology.orchestra.run.smithi031.stdout:999 read 0x4cb7b46 thru 0x4cc7629 (0xfae4 bytes) 2024-09-13T09:07:41.077 INFO:teuthology.orchestra.run.smithi031.stdout:1000 write 0x1ce8f0f thru 0x1cf413f (0xb231 bytes) 2024-09-13T09:07:41.079 INFO:teuthology.orchestra.run.smithi031.stdout:1001 punch from 0xa1ce557 to 0xa1db840, (0xd2e9 bytes) 2024-09-13T09:07:41.080 INFO:teuthology.orchestra.run.smithi031.stdout:1002 trunc from 0xa3b9fac to 0xa8899bc 2024-09-13T09:07:41.082 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:41.082 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:41.088 INFO:teuthology.orchestra.run.smithi031.stdout:1004 writesame 0x997d1e4 thru 0x9981b39 data_size 0x3f(0x4956 bytes) 2024-09-13T09:07:41.093 INFO:teuthology.orchestra.run.smithi031.stdout:1005 write 0x2354b6e thru 0x2360530 (0xb9c3 bytes) 2024-09-13T09:07:41.099 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.101 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:41.101 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:41.113 INFO:teuthology.orchestra.run.smithi031.stdout:1009 write 0xecce471 thru 0xecdba92 (0xd622 bytes) 2024-09-13T09:07:41.117 INFO:teuthology.orchestra.run.smithi031.stdout:1010 read 0x19d6fa5 thru 0x19dd19f (0x61fb bytes) 2024-09-13T09:07:41.118 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.124 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:41.125 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:41.136 INFO:teuthology.orchestra.run.smithi031.stdout:1014 trunc from 0xecdba93 to 0x3fc5774 2024-09-13T09:07:41.150 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:41.151 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:41.151 INFO:teuthology.orchestra.run.smithi031.stdout:1016 read 0xd75911 thru 0xd7fba2 (0xa292 bytes) 2024-09-13T09:07:41.152 INFO:teuthology.orchestra.run.smithi031.stdout:1018 read 0x19355af thru 0x194295d (0xd3af bytes) 2024-09-13T09:07:41.160 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:41.156+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:41.163 INFO:teuthology.orchestra.run.smithi031.stdout:1019 trunc from 0x3fc5774 to 0xc720062 2024-09-13T09:07:41.177 INFO:teuthology.orchestra.run.smithi031.stdout:1021 writesame 0xd054755 thru 0xd05d130 data_size 0x22(0x89dc bytes) 2024-09-13T09:07:41.190 INFO:teuthology.orchestra.run.smithi031.stdout:1022 trunc from 0xd05d131 to 0x2d5e84a 2024-09-13T09:07:41.208 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:41.208 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:41.215 INFO:teuthology.orchestra.run.smithi031.stdout:1023 write 0xa491191 thru 0xa493a27 (0x2897 bytes) 2024-09-13T09:07:41.218 INFO:teuthology.orchestra.run.smithi031.stdout:1024 writesame 0x8e40a5f thru 0x8e41f24 data_size 0x1(0x14c6 bytes) 2024-09-13T09:07:41.223 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:41.235 INFO:teuthology.orchestra.run.smithi031.stdout:1027 write 0xe763b98 thru 0xe7730b0 (0xf519 bytes) 2024-09-13T09:07:41.239 INFO:teuthology.orchestra.run.smithi031.stdout:1028 punch from 0x3e5f983 to 0x3e630f2, (0x376f bytes) 2024-09-13T09:07:41.240 INFO:teuthology.orchestra.run.smithi031.stdout:1029 write 0x1794cc2 thru 0x17a1229 (0xc568 bytes) 2024-09-13T09:07:41.249 INFO:teuthology.orchestra.run.smithi031.stdout:1030 trunc from 0xe7730b1 to 0x8487b2a 2024-09-13T09:07:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.268 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:41.266+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:1033 trunc from 0x8487b2a to 0x776627d 2024-09-13T09:07:41.279 INFO:teuthology.orchestra.run.smithi031.stdout:1034 punch from 0x1f35cd1 to 0x1f42631, (0xc960 bytes) 2024-09-13T09:07:41.280 INFO:teuthology.orchestra.run.smithi031.stdout:1035 read 0x38bf9e3 thru 0x38c9ccf (0xa2ed bytes) 2024-09-13T09:07:41.290 INFO:teuthology.orchestra.run.smithi031.stdout:1036 write 0x90f5bcc thru 0x90fb22e (0x5663 bytes) 2024-09-13T09:07:41.293 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.293 INFO:teuthology.orchestra.run.smithi031.stdout:1039 read 0x8ac00bf thru 0x8acd968 (0xd8aa bytes) 2024-09-13T09:07:41.293 INFO:teuthology.orchestra.run.smithi031.stdout:1040 punch from 0xbd11ff to 0xbdab21, (0x9922 bytes) 2024-09-13T09:07:41.293 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.293 INFO:teuthology.orchestra.run.smithi031.stdout:1042 read 0x7924fde thru 0x792c732 (0x7755 bytes) 2024-09-13T09:07:41.294 INFO:teuthology.orchestra.run.smithi031.stdout:1043 read 0x6b0afb3 thru 0x6b0b2d0 (0x31e bytes) 2024-09-13T09:07:41.297 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.297 INFO:teuthology.orchestra.run.smithi031.stdout:1047 writesame 0x6e6aeae thru 0x6e75df5 data_size 0x4f(0xaf48 bytes) 2024-09-13T09:07:41.299 INFO:teuthology.orchestra.run.smithi031.stdout:1049 read 0x5f8c945 thru 0x5f97124 (0xa7e0 bytes) 2024-09-13T09:07:41.300 INFO:teuthology.orchestra.run.smithi031.stdout:1050 punch from 0x7093342 to 0x7096d13, (0x39d1 bytes) 2024-09-13T09:07:41.315 INFO:teuthology.orchestra.run.smithi031.stdout:1051 write 0xc1d46b0 thru 0xc1d79e5 (0x3336 bytes) 2024-09-13T09:07:41.333 INFO:teuthology.orchestra.run.smithi031.stdout:1052 write 0xdd64de1 thru 0xdd6ba8f (0x6caf bytes) 2024-09-13T09:07:41.336 INFO:teuthology.orchestra.run.smithi031.stdout:1053 write 0x39da16a thru 0x39e7fc8 (0xde5f bytes) 2024-09-13T09:07:41.339 INFO:teuthology.orchestra.run.smithi031.stdout:1054 write 0xca15e23 thru 0xca18e4c (0x302a bytes) 2024-09-13T09:07:41.345 INFO:teuthology.orchestra.run.smithi031.stdout:1056 read 0x4f80428 thru 0x4f89b46 (0x971f bytes) 2024-09-13T09:07:41.346 INFO:teuthology.orchestra.run.smithi031.stdout:1057 writesame 0xc87a161 thru 0xc87c913 data_size 0x1(0x27b3 bytes) 2024-09-13T09:07:41.350 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:41.350 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:41.363 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:41.360+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:41.363 INFO:teuthology.orchestra.run.smithi031.stdout:1061 writesame 0x74982e7 thru 0x749fad6 data_size 0x65(0x77f0 bytes) 2024-09-13T09:07:41.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:41.367 INFO:teuthology.orchestra.run.smithi031.stdout:1062 read 0x9002b48 thru 0x9008945 (0x5dfe bytes) 2024-09-13T09:07:41.368 INFO:teuthology.orchestra.run.smithi031.stdout:1063 read 0xcb3aefd thru 0xcb43845 (0x8949 bytes) 2024-09-13T09:07:41.369 INFO:teuthology.orchestra.run.smithi031.stdout:1064 write 0x48f5433 thru 0x48ff771 (0xa33f bytes) 2024-09-13T09:07:41.373 INFO:teuthology.orchestra.run.smithi031.stdout:1065 writesame 0x92f648d thru 0x9305c41 data_size 0x1(0xf7b5 bytes) 2024-09-13T09:07:41.463 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:41.460+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:41.508 INFO:teuthology.orchestra.run.smithi031.stdout:1066 write 0xe8b96e0 thru 0xe8c0bc5 (0x74e6 bytes) 2024-09-13T09:07:41.519 INFO:teuthology.orchestra.run.smithi031.stdout:1067 trunc from 0xe8c0bc6 to 0xcbe875c 2024-09-13T09:07:41.537 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:41.537 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:1069 punch from 0x43f55b5 to 0x43f60ee, (0xb39 bytes) 2024-09-13T09:07:41.554 INFO:teuthology.orchestra.run.smithi031.stdout:1070 read 0x883b63 thru 0x89115e (0xd5fc bytes) 2024-09-13T09:07:41.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:41.556 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.556 INFO:teuthology.orchestra.run.smithi031.stdout:1074 punch from 0x8eda458 to 0x8ee251a, (0x80c2 bytes) 2024-09-13T09:07:41.556 INFO:teuthology.orchestra.run.smithi031.stdout:1075 write 0x1ddc44c thru 0x1de9bb4 (0xd769 bytes) 2024-09-13T09:07:41.558 INFO:teuthology.orchestra.run.smithi031.stdout:1076 write 0x9e5ba3 thru 0x9efb31 (0x9f8f bytes) 2024-09-13T09:07:41.561 INFO:teuthology.orchestra.run.smithi031.stdout:1078 write 0x1682912 thru 0x1688ff4 (0x66e3 bytes) 2024-09-13T09:07:41.566 INFO:teuthology.orchestra.run.smithi031.stdout:1079 write 0x718b384 thru 0x718f36b (0x3fe8 bytes) 2024-09-13T09:07:41.568 INFO:teuthology.orchestra.run.smithi031.stdout:1080 write 0x49ef47e thru 0x49feb09 (0xf68c bytes) 2024-09-13T09:07:41.571 INFO:teuthology.orchestra.run.smithi031.stdout:1082 writesame 0xbf27b27 thru 0xbf2a6cc data_size 0x2(0x2ba6 bytes) 2024-09-13T09:07:41.582 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.601 INFO:teuthology.orchestra.run.smithi031.stdout:1087 writesame 0xed02a9c thru 0xed129d8 data_size 0x13(0xff3d bytes) 2024-09-13T09:07:41.601 INFO:teuthology.orchestra.run.smithi031.stdout:1089 write 0x67dfe68 thru 0x67eee0e (0xefa7 bytes) 2024-09-13T09:07:41.604 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:41.602+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:41.632 INFO:teuthology.orchestra.run.smithi031.stdout:1090 trunc from 0xed129d9 to 0x473012b 2024-09-13T09:07:41.664 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:41.662+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:41.676 INFO:teuthology.orchestra.run.smithi031.stdout:1091 trunc from 0x473012b to 0xddce300 2024-09-13T09:07:41.686 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:41.686 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:41.688 INFO:teuthology.orchestra.run.smithi031.stdout:1094 write 0xda0e636 thru 0xda115ad (0x2f78 bytes) 2024-09-13T09:07:41.691 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.691 INFO:teuthology.orchestra.run.smithi031.stdout:1096 write 0xd660540 thru 0xd664e2d (0x48ee bytes) 2024-09-13T09:07:41.694 INFO:teuthology.orchestra.run.smithi031.stdout:1097 writesame 0x24522c6 thru 0x245bd5f data_size 0x16(0x9a9a bytes) 2024-09-13T09:07:41.700 INFO:teuthology.orchestra.run.smithi031.stdout:1098 write 0xb48c4c1 thru 0xb48eba0 (0x26e0 bytes) 2024-09-13T09:07:41.701 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:41.703 INFO:teuthology.orchestra.run.smithi031.stdout:1101 write 0x98d110a thru 0x98e0c4c (0xfb43 bytes) 2024-09-13T09:07:41.710 INFO:teuthology.orchestra.run.smithi031.stdout:1102 punch from 0xd8f3f59 to 0xd8ff40e, (0xb4b5 bytes) 2024-09-13T09:07:41.710 INFO:teuthology.orchestra.run.smithi031.stdout:1105 read 0xca1d3a5 thru 0xca1dd65 (0x9c1 bytes) 2024-09-13T09:07:41.710 INFO:teuthology.orchestra.run.smithi031.stdout:1106 writesame 0xd52ccd9 thru 0xd53ab46 data_size 0x47(0xde6e bytes) 2024-09-13T09:07:41.711 INFO:teuthology.orchestra.run.smithi031.stdout:1107 read 0xd1f1ede thru 0xd1f5554 (0x3677 bytes) 2024-09-13T09:07:41.712 INFO:teuthology.orchestra.run.smithi031.stdout:1108 read 0x49ae323 thru 0x49b236e (0x404c bytes) 2024-09-13T09:07:41.713 INFO:teuthology.orchestra.run.smithi031.stdout:1109 read 0x7cc6ea0 thru 0x7cce775 (0x78d6 bytes) 2024-09-13T09:07:41.714 INFO:teuthology.orchestra.run.smithi031.stdout:1110 read 0x9afa9a3 thru 0x9afd25e (0x28bc bytes) 2024-09-13T09:07:41.715 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.716 INFO:teuthology.orchestra.run.smithi031.stdout:1112 trunc from 0xddce300 to 0xe6e45b7 2024-09-13T09:07:41.730 INFO:teuthology.orchestra.run.smithi031.stdout:1113 read 0x823adc3 thru 0x823da21 (0x2c5f bytes) 2024-09-13T09:07:41.730 INFO:teuthology.orchestra.run.smithi031.stdout:1117 read 0x1bdf7d8 thru 0x1be2dc4 (0x35ed bytes) 2024-09-13T09:07:41.731 INFO:teuthology.orchestra.run.smithi031.stdout:1118 write 0x1451608 thru 0x145be88 (0xa881 bytes) 2024-09-13T09:07:41.734 INFO:teuthology.orchestra.run.smithi031.stdout:1119 read 0xcc0aa55 thru 0xcc11fd1 (0x757d bytes) 2024-09-13T09:07:41.736 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.736 INFO:teuthology.orchestra.run.smithi031.stdout:1121 writesame 0xae01ba5 thru 0xae0da53 data_size 0xd(0xbeaf bytes) 2024-09-13T09:07:41.762 INFO:teuthology.orchestra.run.smithi031.stdout:1122 trunc from 0xe6e45b7 to 0x81e722c 2024-09-13T09:07:41.766 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:41.764+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:41.797 INFO:teuthology.orchestra.run.smithi031.stdout:1123 trunc from 0x81e722c to 0xe0be9b3 2024-09-13T09:07:41.816 INFO:teuthology.orchestra.run.smithi031.stdout:1124 trunc from 0xe0be9b3 to 0xa735e9f 2024-09-13T09:07:41.851 INFO:teuthology.orchestra.run.smithi031.stdout:1125 write 0xcff75f4 thru 0xd00746d (0xfe7a bytes) 2024-09-13T09:07:41.854 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.854 INFO:teuthology.orchestra.run.smithi031.stdout:1127 read 0xcec3319 thru 0xcec8ecd (0x5bb5 bytes) 2024-09-13T09:07:41.855 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:41.856 INFO:teuthology.orchestra.run.smithi031.stdout:1129 write 0x49c2361 thru 0x49cc2f5 (0x9f95 bytes) 2024-09-13T09:07:41.858 INFO:teuthology.orchestra.run.smithi031.stdout:1130 write 0x4cc44f thru 0x4cdc60 (0x1812 bytes) 2024-09-13T09:07:41.867 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:41.862+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:41.879 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:41.879 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:41.891 INFO:teuthology.orchestra.run.smithi031.stdout:1131 trunc from 0xd00746e to 0x28cb5c4 2024-09-13T09:07:41.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:1132 read 0x199cb40 thru 0x19ac743 (0xfc04 bytes) 2024-09-13T09:07:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:1134 read 0x19e7c3b thru 0x19f56b2 (0xda78 bytes) 2024-09-13T09:07:41.909 INFO:teuthology.orchestra.run.smithi031.stdout:1137 read 0x1fcb9fb thru 0x1fd935f (0xd965 bytes) 2024-09-13T09:07:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:1138 write 0x1006f91 thru 0x10085c9 (0x1639 bytes) 2024-09-13T09:07:41.912 INFO:teuthology.orchestra.run.smithi031.stdout:1139 read 0x27f132a thru 0x27f3d74 (0x2a4b bytes) 2024-09-13T09:07:41.913 INFO:teuthology.orchestra.run.smithi031.stdout:1140 read 0x1b19326 thru 0x1b1d8ec (0x45c7 bytes) 2024-09-13T09:07:41.914 INFO:teuthology.orchestra.run.smithi031.stdout:1141 punch from 0xc0cd0d to 0xc0ff34, (0x3227 bytes) 2024-09-13T09:07:41.914 INFO:teuthology.orchestra.run.smithi031.stdout:1143 punch from 0x273b7df to 0x273e131, (0x2952 bytes) 2024-09-13T09:07:41.934 INFO:teuthology.orchestra.run.smithi031.stdout:1144 write 0x55aa175 thru 0x55ae488 (0x4314 bytes) 2024-09-13T09:07:41.938 INFO:teuthology.orchestra.run.smithi031.stdout:1145 write 0x47a55b thru 0x484584 (0xa02a bytes) 2024-09-13T09:07:41.941 INFO:teuthology.orchestra.run.smithi031.stdout:1148 read 0xff563e thru 0x10042ea (0xecad bytes) 2024-09-13T09:07:41.978 INFO:teuthology.orchestra.run.smithi031.stdout:1150 writesame 0xd90769f thru 0xd90edae data_size 0x78(0x7710 bytes) 2024-09-13T09:07:41.993 INFO:teuthology.orchestra.run.smithi031.stdout:1151 write 0xdb52f09 thru 0xdb559d3 (0x2acb bytes) 2024-09-13T09:07:41.996 INFO:teuthology.orchestra.run.smithi031.stdout:1152 write 0xc14188c thru 0xc147421 (0x5b96 bytes) 2024-09-13T09:07:42.000 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.000 INFO:teuthology.orchestra.run.smithi031.stdout:1155 read 0x6480804 thru 0x6489238 (0x8a35 bytes) 2024-09-13T09:07:42.001 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.001 INFO:teuthology.orchestra.run.smithi031.stdout:1158 write 0xd15bb94 thru 0xd15e1f4 (0x2661 bytes) 2024-09-13T09:07:42.001 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:41.998+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:42.004 INFO:teuthology.orchestra.run.smithi031.stdout:1159 read 0xd031a2b thru 0xd0389f0 (0x6fc6 bytes) 2024-09-13T09:07:42.013 INFO:teuthology.orchestra.run.smithi031.stdout:1162 trunc from 0xdb559d4 to 0xb006d27 2024-09-13T09:07:42.055 INFO:teuthology.orchestra.run.smithi031.stdout:1163 trunc from 0xb006d27 to 0x221d4df 2024-09-13T09:07:42.070 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:42.064+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:42.074 INFO:teuthology.orchestra.run.smithi031.stdout:1164 read 0x19b6390 thru 0x19c09de (0xa64f bytes) 2024-09-13T09:07:42.088 INFO:teuthology.orchestra.run.smithi031.stdout:1165 write 0x2352648 thru 0x235e50e (0xbec7 bytes) 2024-09-13T09:07:42.091 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:42.091 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:42.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:42.112 INFO:teuthology.orchestra.run.smithi031.stdout:1168 write 0x597f237 thru 0x5983036 (0x3e00 bytes) 2024-09-13T09:07:42.115 INFO:teuthology.orchestra.run.smithi031.stdout:1169 punch from 0x551445b to 0x55226c6, (0xe26b bytes) 2024-09-13T09:07:42.115 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.137 INFO:teuthology.orchestra.run.smithi031.stdout:1171 writesame 0x94accdb thru 0x94b380b data_size 0x3(0x6b31 bytes) 2024-09-13T09:07:42.157 INFO:teuthology.orchestra.run.smithi031.stdout:1172 trunc from 0x94b380c to 0x1e4ef78 2024-09-13T09:07:42.173 INFO:teuthology.orchestra.run.smithi031.stdout:1173 read 0x19d696e thru 0x19d9379 (0x2a0c bytes) 2024-09-13T09:07:42.192 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:42.190+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:42.197 INFO:teuthology.orchestra.run.smithi031.stdout:1174 writesame 0x601642b thru 0x601d79d data_size 0x17(0x7373 bytes) 2024-09-13T09:07:42.202 INFO:teuthology.orchestra.run.smithi031.stdout:1175 trunc from 0x601d79e to 0x745ad02 2024-09-13T09:07:42.213 INFO:teuthology.orchestra.run.smithi031.stdout:1176 punch from 0x59896b6 to 0x598cf58, (0x38a2 bytes) 2024-09-13T09:07:42.213 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.234 INFO:teuthology.orchestra.run.smithi031.stdout:1179 write 0xaa81e80 thru 0xaa90e1c (0xef9d bytes) 2024-09-13T09:07:42.252 INFO:teuthology.orchestra.run.smithi031.stdout:1180 write 0xb2991f6 thru 0xb2a4a51 (0xb85c bytes) 2024-09-13T09:07:42.255 INFO:teuthology.orchestra.run.smithi031.stdout:1181 read 0x5bda92f thru 0x5bdecec (0x43be bytes) 2024-09-13T09:07:42.256 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.256 INFO:teuthology.orchestra.run.smithi031.stdout:1183 read 0x35c5ce4 thru 0x35d2d44 (0xd061 bytes) 2024-09-13T09:07:42.257 INFO:teuthology.orchestra.run.smithi031.stdout:1184 punch from 0x560f2b to 0x56dee8, (0xcfbd bytes) 2024-09-13T09:07:42.257 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.257 INFO:teuthology.orchestra.run.smithi031.stdout:1187 write 0x4aae8dd thru 0x4ab4f9d (0x66c1 bytes) 2024-09-13T09:07:42.259 INFO:teuthology.orchestra.run.smithi031.stdout:1188 read 0x912e882 thru 0x9135e67 (0x75e6 bytes) 2024-09-13T09:07:42.260 INFO:teuthology.orchestra.run.smithi031.stdout:1189 read 0x209b748 thru 0x20a2718 (0x6fd1 bytes) 2024-09-13T09:07:42.261 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.261 INFO:teuthology.orchestra.run.smithi031.stdout:1191 read 0x5ec3d0 thru 0x5f607b (0x9cac bytes) 2024-09-13T09:07:42.262 INFO:teuthology.orchestra.run.smithi031.stdout:1192 write 0x1a7aa71 thru 0x1a7b6a7 (0xc37 bytes) 2024-09-13T09:07:42.271 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:42.266+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:42.280 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:42.281 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:42.283 INFO:teuthology.orchestra.run.smithi031.stdout:1193 write 0xc74df0f thru 0xc757522 (0x9614 bytes) 2024-09-13T09:07:42.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:42.302 INFO:teuthology.orchestra.run.smithi031.stdout:1194 write 0xdf27829 thru 0xdf3704d (0xf825 bytes) 2024-09-13T09:07:42.305 INFO:teuthology.orchestra.run.smithi031.stdout:1196 read 0x12dda22 thru 0x12df2c0 (0x189f bytes) 2024-09-13T09:07:42.306 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.306 INFO:teuthology.orchestra.run.smithi031.stdout:1199 write 0xdba4f0 thru 0xdc0fcb (0x6adc bytes) 2024-09-13T09:07:42.308 INFO:teuthology.orchestra.run.smithi031.stdout:1202 write 0x7656fda thru 0x765ad78 (0x3d9f bytes) 2024-09-13T09:07:42.311 INFO:teuthology.orchestra.run.smithi031.stdout:1205 write 0xd30c689 thru 0xd31a365 (0xdcdd bytes) 2024-09-13T09:07:42.327 INFO:teuthology.orchestra.run.smithi031.stdout:1207 write 0xe42f8f8 thru 0xe43c21a (0xc923 bytes) 2024-09-13T09:07:42.332 INFO:teuthology.orchestra.run.smithi031.stdout:1208 read 0x6f2beca thru 0x6f38ec3 (0xcffa bytes) 2024-09-13T09:07:42.333 INFO:teuthology.orchestra.run.smithi031.stdout:1210 writesame 0x2b89078 thru 0x2b95425 data_size 0x45(0xc3ae bytes) 2024-09-13T09:07:42.339 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.339 INFO:teuthology.orchestra.run.smithi031.stdout:1214 write 0x545b478 thru 0x545f904 (0x448d bytes) 2024-09-13T09:07:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:1215 punch from 0x823f29 to 0x8296ae, (0x5785 bytes) 2024-09-13T09:07:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:1216 punch from 0x9302c41 to 0x930a761, (0x7b20 bytes) 2024-09-13T09:07:42.342 INFO:teuthology.orchestra.run.smithi031.stdout:1217 read 0x8b01ea5 thru 0x8b08f50 (0x70ac bytes) 2024-09-13T09:07:42.343 INFO:teuthology.orchestra.run.smithi031.stdout:1218 writesame 0x838e6d8 thru 0x839ac54 data_size 0x1(0xc57d bytes) 2024-09-13T09:07:42.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:07:42.391 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:42.388+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:42.413 INFO:teuthology.orchestra.run.smithi031.stdout:1219 punch from 0x737ed8a to 0x73829e4, (0x3c5a bytes) 2024-09-13T09:07:42.413 INFO:teuthology.orchestra.run.smithi031.stdout:1220 read 0xabd4650 thru 0xabe255b (0xdf0c bytes) 2024-09-13T09:07:42.414 INFO:teuthology.orchestra.run.smithi031.stdout:1221 writesame 0xba742a2 thru 0xba753a5 data_size 0x21(0x1104 bytes) 2024-09-13T09:07:42.419 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.419 INFO:teuthology.orchestra.run.smithi031.stdout:1223 read 0xc3d4d76 thru 0xc3d5eb7 (0x1142 bytes) 2024-09-13T09:07:42.419 INFO:teuthology.orchestra.run.smithi031.stdout:1224 write 0xb59a32 thru 0xb5eb7c (0x514b bytes) 2024-09-13T09:07:42.422 INFO:teuthology.orchestra.run.smithi031.stdout:1225 read 0x70b5c92 thru 0x70b6e78 (0x11e7 bytes) 2024-09-13T09:07:42.423 INFO:teuthology.orchestra.run.smithi031.stdout:1226 write 0x615c577 thru 0x6160ca3 (0x472d bytes) 2024-09-13T09:07:42.426 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.426 INFO:teuthology.orchestra.run.smithi031.stdout:1230 read 0x4b4a1b9 thru 0x4b4ade5 (0xc2d bytes) 2024-09-13T09:07:42.427 INFO:teuthology.orchestra.run.smithi031.stdout:1231 writesame 0x9a8d323 thru 0x9a8df4c data_size 0x9(0xc2a bytes) 2024-09-13T09:07:42.429 INFO:teuthology.orchestra.run.smithi031.stdout:1232 write 0x3afc8fd thru 0x3b02908 (0x600c bytes) 2024-09-13T09:07:42.433 INFO:teuthology.orchestra.run.smithi031.stdout:1233 trunc from 0xe43c21b to 0xdb736b2 2024-09-13T09:07:42.463 INFO:teuthology.orchestra.run.smithi031.stdout:1234 punch from 0x1cac3ee to 0x1cbc3d7, (0xffe9 bytes) 2024-09-13T09:07:42.464 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:42.465 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:42.473 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:42.468+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:42.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:42.482 INFO:teuthology.orchestra.run.smithi031.stdout:1235 trunc from 0xdb736b2 to 0x161d27d 2024-09-13T09:07:42.497 INFO:teuthology.orchestra.run.smithi031.stdout:1236 read 0x13608b2 thru 0x1368821 (0x7f70 bytes) 2024-09-13T09:07:42.497 INFO:teuthology.orchestra.run.smithi031.stdout:1237 read 0xbf95c5 thru 0xbfca99 (0x34d5 bytes) 2024-09-13T09:07:42.497 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.497 INFO:teuthology.orchestra.run.smithi031.stdout:1239 punch from 0xdb1f92 to 0xdc18ce, (0xf93c bytes) 2024-09-13T09:07:42.507 INFO:teuthology.orchestra.run.smithi031.stdout:1241 trunc from 0x161d27d to 0x7c8a4c1 2024-09-13T09:07:42.522 INFO:teuthology.orchestra.run.smithi031.stdout:1243 write 0x8c64690 thru 0x8c66df1 (0x2762 bytes) 2024-09-13T09:07:42.523 INFO:teuthology.orchestra.run.smithi031.stdout:1244 write 0x32b03b3 thru 0x32b77a3 (0x73f1 bytes) 2024-09-13T09:07:42.524 INFO:teuthology.orchestra.run.smithi031.stdout:1245 read 0x5b71d63 thru 0x5b8019f (0xe43d bytes) 2024-09-13T09:07:42.525 INFO:teuthology.orchestra.run.smithi031.stdout:1246 write 0x6474613 thru 0x647c2c5 (0x7cb3 bytes) 2024-09-13T09:07:42.535 INFO:teuthology.orchestra.run.smithi031.stdout:1249 trunc from 0x8c66df2 to 0xe4cb15a 2024-09-13T09:07:42.541 INFO:teuthology.orchestra.run.smithi031.stdout:1250 write 0xc874d35 thru 0xc8775b0 (0x287c bytes) 2024-09-13T09:07:42.543 INFO:teuthology.orchestra.run.smithi031.stdout:1251 read 0xd2df24f thru 0xd2e6706 (0x74b8 bytes) 2024-09-13T09:07:42.543 INFO:teuthology.orchestra.run.smithi031.stdout:1252 read 0xd7db41b thru 0xd7e536f (0x9f55 bytes) 2024-09-13T09:07:42.544 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.563 INFO:teuthology.orchestra.run.smithi031.stdout:1254 trunc from 0xe4cb15a to 0xd88cad 2024-09-13T09:07:42.574 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:42.570+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:42.589 INFO:teuthology.orchestra.run.smithi031.stdout:1255 write 0x3db1498 thru 0x3dbfefd (0xea66 bytes) 2024-09-13T09:07:42.609 INFO:teuthology.orchestra.run.smithi031.stdout:1256 writesame 0x9d16957 thru 0x9d21914 data_size 0x37(0xafbe bytes) 2024-09-13T09:07:42.613 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:42.613 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:42.625 INFO:teuthology.orchestra.run.smithi031.stdout:1257 trunc from 0x9d21915 to 0x16fa4fe 2024-09-13T09:07:42.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:42.650 INFO:teuthology.orchestra.run.smithi031.stdout:1259 write 0x2ae52fe thru 0x2af1f82 (0xcc85 bytes) 2024-09-13T09:07:42.653 INFO:teuthology.orchestra.run.smithi031.stdout:1260 write 0x435c31 thru 0x43d649 (0x7a19 bytes) 2024-09-13T09:07:42.655 INFO:teuthology.orchestra.run.smithi031.stdout:1262 read 0xab9aba thru 0xac1ee5 (0x842c bytes) 2024-09-13T09:07:42.657 INFO:teuthology.orchestra.run.smithi031.stdout:1263 trunc from 0x2af1f83 to 0x20b716b 2024-09-13T09:07:42.672 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.680 INFO:teuthology.orchestra.run.smithi031.stdout:1266 trunc from 0x20b716b to 0x668984c 2024-09-13T09:07:42.689 INFO:teuthology.orchestra.run.smithi031.stdout:1267 read 0x3c134ae thru 0x3c1ee0f (0xb962 bytes) 2024-09-13T09:07:42.696 INFO:teuthology.orchestra.run.smithi031.stdout:1269 trunc from 0x668984c to 0x3248c4b 2024-09-13T09:07:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.709 INFO:teuthology.orchestra.run.smithi031.stdout:1275 read 0xf60f7b thru 0xf649b7 (0x3a3d bytes) 2024-09-13T09:07:42.716 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:42.714+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:42.729 INFO:teuthology.orchestra.run.smithi031.stdout:1276 write 0x7f33f34 thru 0x7f3d23e (0x930b bytes) 2024-09-13T09:07:42.748 INFO:teuthology.orchestra.run.smithi031.stdout:1277 trunc from 0x7f3d23f to 0x9a9fba 2024-09-13T09:07:42.767 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:07:42.775 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:42.772+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:42.783 INFO:teuthology.orchestra.run.smithi031.stdout:1279 read 0x479960 thru 0x482622 (0x8cc3 bytes) 2024-09-13T09:07:42.789 INFO:teuthology.orchestra.run.smithi031.stdout:1281 trunc from 0x9a9fba to 0x38bace0 2024-09-13T09:07:42.810 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:42.810 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:42.812 INFO:teuthology.orchestra.run.smithi031.stdout:1282 writesame 0x76812ef thru 0x7682550 data_size 0xd(0x1262 bytes) 2024-09-13T09:07:42.813 INFO:teuthology.orchestra.run.smithi031.stdout:1284 read 0x5089ecb thru 0x508b2cb (0x1401 bytes) 2024-09-13T09:07:42.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:42.830 INFO:teuthology.orchestra.run.smithi031.stdout:1285 write 0xb8c8e50 thru 0xb8d27b5 (0x9966 bytes) 2024-09-13T09:07:42.833 INFO:teuthology.orchestra.run.smithi031.stdout:1286 punch from 0x5006124 to 0x5015ce0, (0xfbbc bytes) 2024-09-13T09:07:42.833 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:42.833 INFO:teuthology.orchestra.run.smithi031.stdout:1290 punch from 0xacea055 to 0xacf5749, (0xb6f4 bytes) 2024-09-13T09:07:42.833 INFO:teuthology.orchestra.run.smithi031.stdout:1291 read 0x43f479f thru 0x43fddd4 (0x9636 bytes) 2024-09-13T09:07:42.833 INFO:teuthology.orchestra.run.smithi031.stdout:1293 write 0x61229a8 thru 0x612778a (0x4de3 bytes) 2024-09-13T09:07:42.854 INFO:teuthology.orchestra.run.smithi031.stdout:1295 write 0xdb49707 thru 0xdb58e27 (0xf721 bytes) 2024-09-13T09:07:42.858 INFO:teuthology.orchestra.run.smithi031.stdout:1296 punch from 0xb4912ee to 0xb49b690, (0xa3a2 bytes) 2024-09-13T09:07:42.858 INFO:teuthology.orchestra.run.smithi031.stdout:1298 punch from 0x23b3c56 to 0x23beeba, (0xb264 bytes) 2024-09-13T09:07:42.858 INFO:teuthology.orchestra.run.smithi031.stdout:1299 write 0xb5c8a9a thru 0xb5d18bb (0x8e22 bytes) 2024-09-13T09:07:42.860 INFO:teuthology.orchestra.run.smithi031.stdout:1300 read 0x21ae549 thru 0x21bd5c7 (0xf07f bytes) 2024-09-13T09:07:42.861 INFO:teuthology.orchestra.run.smithi031.stdout:1301 writesame 0x5e8b5d4 thru 0x5e8c3c3 data_size 0x4(0xdf0 bytes) 2024-09-13T09:07:42.867 INFO:teuthology.orchestra.run.smithi031.stdout:1303 punch from 0x5195a3a to 0x51a3f0e, (0xe4d4 bytes) 2024-09-13T09:07:42.895 INFO:teuthology.orchestra.run.smithi031.stdout:1304 trunc from 0xdb58e28 to 0x2b020a7 2024-09-13T09:07:42.910 INFO:teuthology.orchestra.run.smithi031.stdout:1305 read 0x1215961 thru 0x12251aa (0xf84a bytes) 2024-09-13T09:07:42.916 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:42.914+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:42.943 INFO:teuthology.orchestra.run.smithi031.stdout:1306 writesame 0xb8ea4da thru 0xb8eed8f data_size 0x2(0x48b6 bytes) 2024-09-13T09:07:42.946 INFO:teuthology.orchestra.run.smithi031.stdout:1307 punch from 0x29520b to 0x29a130, (0x4f25 bytes) 2024-09-13T09:07:42.946 INFO:teuthology.orchestra.run.smithi031.stdout:1308 write 0x58a3cc5 thru 0x58a9444 (0x5780 bytes) 2024-09-13T09:07:42.948 INFO:teuthology.orchestra.run.smithi031.stdout:1310 read 0x81d3aa3 thru 0x81d3f52 (0x4b0 bytes) 2024-09-13T09:07:42.949 INFO:teuthology.orchestra.run.smithi031.stdout:1311 read 0x3bcc6ab thru 0x3bd4e1e (0x8774 bytes) 2024-09-13T09:07:42.966 INFO:teuthology.orchestra.run.smithi031.stdout:1312 writesame 0xd887a73 thru 0xd896c67 data_size 0xb(0xf1f5 bytes) 2024-09-13T09:07:42.969 INFO:teuthology.orchestra.run.smithi031.stdout:1314 punch from 0xb88e822 to 0xb898595, (0x9d73 bytes) 2024-09-13T09:07:42.977 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:42.974+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:43.000 INFO:teuthology.orchestra.run.smithi031.stdout:1316 trunc from 0xd896c68 to 0x9f2086 2024-09-13T09:07:43.001 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:43.001 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:43.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:43.043 INFO:teuthology.orchestra.run.smithi031.stdout:1318 trunc from 0x9f2086 to 0x65e5fed 2024-09-13T09:07:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:1320 read 0x16b4528 thru 0x16b64cf (0x1fa8 bytes) 2024-09-13T09:07:43.056 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:43.078 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:43.072+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:43.086 INFO:teuthology.orchestra.run.smithi031.stdout:1322 write 0xcf1e4a2 thru 0xcf24464 (0x5fc3 bytes) 2024-09-13T09:07:43.092 INFO:teuthology.orchestra.run.smithi031.stdout:1323 read 0x833f25a thru 0x834bdee (0xcb95 bytes) 2024-09-13T09:07:43.093 INFO:teuthology.orchestra.run.smithi031.stdout:1325 punch from 0x9eaeef0 to 0x9eaf776, (0x886 bytes) 2024-09-13T09:07:43.093 INFO:teuthology.orchestra.run.smithi031.stdout:1326 writesame 0x6d70642 thru 0x6d78313 data_size 0xd(0x7cd2 bytes) 2024-09-13T09:07:43.134 INFO:teuthology.orchestra.run.smithi031.stdout:1327 trunc from 0xcf24465 to 0x799514 2024-09-13T09:07:43.163 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:43.163 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:43.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:43.179 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:43.176+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:43.182 INFO:teuthology.orchestra.run.smithi031.stdout:1332 writesame 0x32140a1 thru 0x321a70d data_size 0x1(0x666d bytes) 2024-09-13T09:07:43.199 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:43.199 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:43.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:43.217 INFO:teuthology.orchestra.run.smithi031.stdout:1333 write 0xcf80e45 thru 0xcf86e98 (0x6054 bytes) 2024-09-13T09:07:43.223 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:43.223 INFO:teuthology.orchestra.run.smithi031.stdout:1336 read 0x43176ac thru 0x431c935 (0x528a bytes) 2024-09-13T09:07:43.242 INFO:teuthology.orchestra.run.smithi031.stdout:1337 trunc from 0xcf86e99 to 0x1946d63 2024-09-13T09:07:43.251 INFO:teuthology.orchestra.run.smithi031.stdout:1338 read 0x184d1ae thru 0x18533b8 (0x620b bytes) 2024-09-13T09:07:43.272 INFO:teuthology.orchestra.run.smithi031.stdout:1339 writesame 0xb5a66ad thru 0xb5a7c82 data_size 0x1a(0x15d6 bytes) 2024-09-13T09:07:43.273 INFO:teuthology.orchestra.run.smithi031.stdout:1341 writesame 0x4adc7cf thru 0x4adc9be data_size 0x4(0x1f0 bytes) 2024-09-13T09:07:43.275 INFO:teuthology.orchestra.run.smithi031.stdout:1342 writesame 0x52abbcd thru 0x52b2bee data_size 0x1f(0x7022 bytes) 2024-09-13T09:07:43.279 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:43.276+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:43.280 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:43.280 INFO:teuthology.orchestra.run.smithi031.stdout:1344 write 0x1781b8d thru 0x1782119 (0x58d bytes) 2024-09-13T09:07:43.285 INFO:teuthology.orchestra.run.smithi031.stdout:1345 read 0xb2390f9 thru 0xb23f892 (0x679a bytes) 2024-09-13T09:07:43.300 INFO:teuthology.orchestra.run.smithi031.stdout:1346 trunc from 0xb5a7c83 to 0x281e7fa 2024-09-13T09:07:43.313 INFO:teuthology.orchestra.run.smithi031.stdout:1347 punch from 0x20210b4 to 0x2022278, (0x11c4 bytes) 2024-09-13T09:07:43.330 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:43.330 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:07:43.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:07:43.333 INFO:teuthology.orchestra.run.smithi031.stdout:1348 write 0x9e4eedf thru 0x9e59983 (0xaaa5 bytes) 2024-09-13T09:07:43.335 INFO:teuthology.orchestra.run.smithi031.stdout:1349 write 0x122fa51 thru 0x123edcb (0xf37b bytes) 2024-09-13T09:07:43.338 INFO:teuthology.orchestra.run.smithi031.stdout:1350 write 0x618e475 thru 0x61900c2 (0x1c4e bytes) 2024-09-13T09:07:43.345 INFO:teuthology.orchestra.run.smithi031.stdout:1351 write 0x6cf7db6 thru 0x6cfb23f (0x348a bytes) 2024-09-13T09:07:43.351 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:43.351 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:43.351 INFO:teuthology.orchestra.run.smithi031.stdout:1352 read 0x5cfbe0a thru 0x5d0b4f6 (0xf6ed bytes) 2024-09-13T09:07:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:43.357 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:43.358 INFO:teuthology.orchestra.run.smithi031.stdout:1353 trunc from 0x9e59984 to 0xd4b2748 2024-09-13T09:07:43.366 INFO:teuthology.orchestra.run.smithi031.stdout:1354 write 0x26d811c thru 0x26d81bd (0xa2 bytes) 2024-09-13T09:07:43.368 INFO:teuthology.orchestra.run.smithi031.stdout:1355 write 0xd12c317 thru 0xd134de2 (0x8acc bytes) 2024-09-13T09:07:43.371 INFO:teuthology.orchestra.run.smithi031.stdout:1356 read 0x16a4dba thru 0x16ac5ad (0x77f4 bytes) 2024-09-13T09:07:43.373 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:43.373 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:43.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:43.375 INFO:teuthology.orchestra.run.smithi031.stdout:1359 trunc from 0xd4b2748 to 0xbab7796 2024-09-13T09:07:43.388 INFO:teuthology.orchestra.run.smithi031.stdout:1360 punch from 0x45d2c91 to 0x45d32ea, (0x659 bytes) 2024-09-13T09:07:43.388 INFO:teuthology.orchestra.run.smithi031.stdout:1362 read 0x97b1667 thru 0x97bac91 (0x962b bytes) 2024-09-13T09:07:43.389 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:43.384+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:43.391 INFO:teuthology.orchestra.run.smithi031.stdout:1364 trunc from 0xbab7796 to 0x998c476 2024-09-13T09:07:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:43.396 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:43.406 INFO:teuthology.orchestra.run.smithi031.stdout:1365 writesame 0x698f3ec thru 0x69921e0 data_size 0x5(0x2df5 bytes) 2024-09-13T09:07:43.411 INFO:teuthology.orchestra.run.smithi031.stdout:1366 read 0xb1402d thru 0xb14aab (0xa7f bytes) 2024-09-13T09:07:43.411 INFO:teuthology.orchestra.run.smithi031.stdout:1367 writesame 0x67d46a8 thru 0x67d734b data_size 0x2(0x2ca4 bytes) 2024-09-13T09:07:43.416 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:43.416 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:43.417 INFO:teuthology.orchestra.run.smithi031.stdout:1368 write 0x7c6e751 thru 0x7c7b126 (0xc9d6 bytes) 2024-09-13T09:07:43.421 INFO:teuthology.orchestra.run.smithi031.stdout:1369 read 0x7cdb326 thru 0x7ce12f0 (0x5fcb bytes) 2024-09-13T09:07:43.422 INFO:teuthology.orchestra.run.smithi031.stdout:1370 writesame 0x6dccde6 thru 0x6dcdaf5 data_size 0x26(0xd10 bytes) 2024-09-13T09:07:43.424 INFO:teuthology.orchestra.run.smithi031.stdout:1371 write 0x5d80802 thru 0x5d8ebe4 (0xe3e3 bytes) 2024-09-13T09:07:43.438 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:43.438 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:43.439 INFO:teuthology.orchestra.run.smithi031.stdout:1372 write 0xc3e3cc3 thru 0xc3e850a (0x4848 bytes) 2024-09-13T09:07:43.442 INFO:teuthology.orchestra.run.smithi031.stdout:1374 read 0x7c7f826 thru 0x7c865ec (0x6dc7 bytes) 2024-09-13T09:07:43.443 INFO:teuthology.orchestra.run.smithi031.stdout:1375 writesame 0x1aee988 thru 0x1af463c data_size 0x9(0x5cb5 bytes) 2024-09-13T09:07:43.448 INFO:teuthology.orchestra.run.smithi031.stdout:1376 read 0x983cc55 thru 0x983eb1a (0x1ec6 bytes) 2024-09-13T09:07:43.452 INFO:teuthology.orchestra.run.smithi031.stdout:1377 trunc from 0xc3e850b to 0xe45870c 2024-09-13T09:07:43.461 INFO:teuthology.orchestra.run.smithi031.stdout:1378 writesame 0x467dfb1 thru 0x4688fec data_size 0x2(0xb03c bytes) 2024-09-13T09:07:43.462 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:43.462 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:43.478 INFO:teuthology.orchestra.run.smithi031.stdout:1379 read 0x53f7aa4 thru 0x53fd91a (0x5e77 bytes) 2024-09-13T09:07:43.479 INFO:teuthology.orchestra.run.smithi031.stdout:1380 read 0xdec4c47 thru 0xdecd9f4 (0x8dae bytes) 2024-09-13T09:07:43.480 INFO:teuthology.orchestra.run.smithi031.stdout:1382 read 0xcf0b05e thru 0xcf101e0 (0x5183 bytes) 2024-09-13T09:07:43.480 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:43.478+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:43.481 INFO:teuthology.orchestra.run.smithi031.stdout:1384 write 0x1a095b1 thru 0x1a17bf0 (0xe640 bytes) 2024-09-13T09:07:43.484 INFO:teuthology.orchestra.run.smithi031.stdout:1385 clone 12 order 20 su 16384 sc 14 2024-09-13T09:07:43.487 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:43.487 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:43.507 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:43.516 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:43.516 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:43.516 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:43.532 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:43.622 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:43.620+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:43.648 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:07:43.649 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:07:43.666 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-09-13T09:07:43.057561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":487236958,"num_objects":406,"num_object_clones":118,"num_object_copies":812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":406,"num_whiteouts":127,"num_read":35682,"num_read_kb":242953,"num_write":3784,"num_write_kb":143680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":108,"num_bytes_recovered":21421855,"num_keys_recovered":16,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2676,"ondisk_log_size":2676,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":430808,"kb_used_data":196352,"kb_used_omap":10,"kb_used_meta":234101,"kb_avail":660172072,"statfs":{"total":676457349120,"available":676016201728,"internally_reserved":0,"allocated":201064448,"data_stored":212793803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11150,"internal_metadata":239719538},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,66,75],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":55873570,"num_objects":60,"num_object_clones":22,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":22,"num_read":7361,"num_read_kb":49519,"num_write":657,"num_write_kb":13857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":17968565,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.753512"},"pg_stats":[{"pgid":"3.1f","version":"39'119","reported_seq":262,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T09:07:37.512534+0000","last_change":"2024-09-13T09:07:28.528187+0000","last_active":"2024-09-13T09:07:37.512534+0000","last_peered":"2024-09-13T09:07:37.512534+0000","last_clean":"2024-09-13T09:07:37.512534+0000","last_became_active":"2024-09-13T09:07:28.519097+0000","last_became_peered":"2024-09-13T09:07:28.519097+0000","last_unstale":"2024-09-13T09:07:37.512534+0000","last_undegraded":"2024-09-13T09:07:37.512534+0000","last_fullsized":"2024-09-13T09:07:37.512534+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:08:15.868014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":6135406,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":557,"num_read_kb":581,"num_write":174,"num_write_kb":623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":88868,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"39'73","reported_seq":1438,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T09:07:37.754299+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:07:37.754299+0000","last_peered":"2024-09-13T09:07:37.754299+0000","last_clean":"2024-09-13T09:07:37.754299+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:07:37.754299+0000","last_undegraded":"2024-09-13T09:07:37.754299+0000","last_fullsized":"2024-09-13T09:07:37.754299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":27556686,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":1981,"num_read_kb":7947,"num_write":65,"num_write_kb":3303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"42'95","reported_seq":1690,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:41.153491+0000","last_change":"2024-09-13T09:07:15.879980+0000","last_active":"2024-09-13T09:07:41.153491+0000","last_peered":"2024-09-13T09:07:41.153491+0000","last_clean":"2024-09-13T09:07:41.153491+0000","last_became_active":"2024-09-13T09:07:13.856004+0000","last_became_peered":"2024-09-13T09:07:13.856004+0000","last_unstale":"2024-09-13T09:07:41.153491+0000","last_undegraded":"2024-09-13T09:07:41.153491+0000","last_fullsized":"2024-09-13T09:07:41.153491+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:37.046343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":38660725,"num_objects":21,"num_object_clones":7,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":1601,"num_read_kb":21638,"num_write":188,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"42'31","reported_seq":819,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.620051+0000","last_change":"2024-09-13T09:07:28.250721+0000","last_active":"2024-09-13T09:07:40.620051+0000","last_peered":"2024-09-13T09:07:40.620051+0000","last_clean":"2024-09-13T09:07:40.620051+0000","last_became_active":"2024-09-13T09:07:28.235516+0000","last_became_peered":"2024-09-13T09:07:28.235516+0000","last_unstale":"2024-09-13T09:07:40.620051+0000","last_undegraded":"2024-09-13T09:07:40.620051+0000","last_fullsized":"2024-09-13T09:07:40.620051+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:29:27.914628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":19400567,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":766,"num_read_kb":10584,"num_write":65,"num_write_kb":3700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"41'133","reported_seq":1495,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.194976+0000","last_change":"2024-09-13T09:07:16.108727+0000","last_active":"2024-09-13T09:07:40.194976+0000","last_peered":"2024-09-13T09:07:40.194976+0000","last_clean":"2024-09-13T09:07:40.194976+0000","last_became_active":"2024-09-13T09:07:13.862565+0000","last_became_peered":"2024-09-13T09:07:13.862565+0000","last_unstale":"2024-09-13T09:07:40.194976+0000","last_undegraded":"2024-09-13T09:07:40.194976+0000","last_fullsized":"2024-09-13T09:07:40.194976+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:32.305423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":7509842,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":2027,"num_read_kb":11517,"num_write":205,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"42'49","reported_seq":659,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.421112+0000","last_change":"2024-09-13T09:07:39.508879+0000","last_active":"2024-09-13T09:07:40.421112+0000","last_peered":"2024-09-13T09:07:40.421112+0000","last_clean":"2024-09-13T09:07:40.421112+0000","last_became_active":"2024-09-13T09:07:39.317064+0000","last_became_peered":"2024-09-13T09:07:39.317064+0000","last_unstale":"2024-09-13T09:07:40.421112+0000","last_undegraded":"2024-09-13T09:07:40.421112+0000","last_fullsized":"2024-09-13T09:07:40.421112+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:08.610037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":42049674,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":510,"num_read_kb":12120,"num_write":117,"num_write_kb":15919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":15904911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"42'34","reported_seq":180,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.378131+0000","last_change":"2024-09-13T09:07:28.251265+0000","last_active":"2024-09-13T09:07:40.378131+0000","last_peered":"2024-09-13T09:07:40.378131+0000","last_clean":"2024-09-13T09:07:40.378131+0000","last_became_active":"2024-09-13T09:07:28.240883+0000","last_became_peered":"2024-09-13T09:07:28.240883+0000","last_unstale":"2024-09-13T09:07:40.378131+0000","last_undegraded":"2024-09-13T09:07:40.378131+0000","last_fullsized":"2024-09-13T09:07:40.378131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:44.829975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":450858,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":106,"num_read_kb":562,"num_write":34,"num_write_kb":3688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":12517,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"39'81","reported_seq":383,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T09:07:38.099368+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:07:38.099368+0000","last_peered":"2024-09-13T09:07:38.099368+0000","last_clean":"2024-09-13T09:07:38.099368+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:07:38.099368+0000","last_undegraded":"2024-09-13T09:07:38.099368+0000","last_fullsized":"2024-09-13T09:07:38.099368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":4350214,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":753,"num_read_kb":791,"num_write":93,"num_write_kb":674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"39'67","reported_seq":1967,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T09:07:37.518407+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:07:37.518407+0000","last_peered":"2024-09-13T09:07:37.518407+0000","last_clean":"2024-09-13T09:07:37.518407+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:07:37.518407+0000","last_undegraded":"2024-09-13T09:07:37.518407+0000","last_fullsized":"2024-09-13T09:07:37.518407+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":18988246,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":2437,"num_read_kb":15306,"num_write":119,"num_write_kb":12282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"41'70","reported_seq":557,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.195486+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:07:40.195486+0000","last_peered":"2024-09-13T09:07:40.195486+0000","last_clean":"2024-09-13T09:07:40.195486+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:07:40.195486+0000","last_undegraded":"2024-09-13T09:07:40.195486+0000","last_fullsized":"2024-09-13T09:07:40.195486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":6995183,"num_objects":15,"num_object_clones":6,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":619,"num_read_kb":3820,"num_write":87,"num_write_kb":3124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"42'91","reported_seq":1192,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:41.145915+0000","last_change":"2024-09-13T09:07:15.873331+0000","last_active":"2024-09-13T09:07:41.145915+0000","last_peered":"2024-09-13T09:07:41.145915+0000","last_clean":"2024-09-13T09:07:41.145915+0000","last_became_active":"2024-09-13T09:07:13.866813+0000","last_became_peered":"2024-09-13T09:07:13.866813+0000","last_unstale":"2024-09-13T09:07:41.145915+0000","last_undegraded":"2024-09-13T09:07:41.145915+0000","last_fullsized":"2024-09-13T09:07:41.145915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.439602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":17585256,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":1528,"num_read_kb":13556,"num_write":103,"num_write_kb":10798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"42'114","reported_seq":1916,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:41.215654+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:07:41.215654+0000","last_peered":"2024-09-13T09:07:41.215654+0000","last_clean":"2024-09-13T09:07:41.215654+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:07:41.215654+0000","last_undegraded":"2024-09-13T09:07:41.215654+0000","last_fullsized":"2024-09-13T09:07:41.215654+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":16530439,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":2223,"num_read_kb":17643,"num_write":120,"num_write_kb":3830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.314894+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:07:39.314894+0000","last_peered":"2024-09-13T09:07:39.314894+0000","last_clean":"2024-09-13T09:07:39.314894+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:07:39.314894+0000","last_undegraded":"2024-09-13T09:07:39.314894+0000","last_fullsized":"2024-09-13T09:07:39.314894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"42'60","reported_seq":563,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.334222+0000","last_change":"2024-09-13T09:07:16.108420+0000","last_active":"2024-09-13T09:07:40.334222+0000","last_peered":"2024-09-13T09:07:40.334222+0000","last_clean":"2024-09-13T09:07:40.334222+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:07:40.334222+0000","last_undegraded":"2024-09-13T09:07:40.334222+0000","last_fullsized":"2024-09-13T09:07:40.334222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":4620326,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":866,"num_read_kb":4535,"num_write":58,"num_write_kb":752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.310929+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:07:39.310929+0000","last_peered":"2024-09-13T09:07:39.310929+0000","last_clean":"2024-09-13T09:07:39.310929+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:07:39.310929+0000","last_undegraded":"2024-09-13T09:07:39.310929+0000","last_fullsized":"2024-09-13T09:07:39.310929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"41'51","reported_seq":917,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.087439+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:07:40.087439+0000","last_peered":"2024-09-13T09:07:40.087439+0000","last_clean":"2024-09-13T09:07:40.087439+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:07:40.087439+0000","last_undegraded":"2024-09-13T09:07:40.087439+0000","last_fullsized":"2024-09-13T09:07:40.087439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":9298118,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":880,"num_read_kb":10112,"num_write":50,"num_write_kb":3465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.314478+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:07:39.314478+0000","last_peered":"2024-09-13T09:07:39.314478+0000","last_clean":"2024-09-13T09:07:39.314478+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:07:39.314478+0000","last_undegraded":"2024-09-13T09:07:39.314478+0000","last_fullsized":"2024-09-13T09:07:39.314478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"41'196","reported_seq":504,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.026618+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:07:40.026618+0000","last_peered":"2024-09-13T09:07:40.026618+0000","last_clean":"2024-09-13T09:07:40.026618+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:07:40.026618+0000","last_undegraded":"2024-09-13T09:07:40.026618+0000","last_fullsized":"2024-09-13T09:07:40.026618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":3297330,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":1246,"num_read_kb":3356,"num_write":346,"num_write_kb":676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T09:07:34.288205+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:07:34.288205+0000","last_peered":"2024-09-13T09:07:34.288205+0000","last_clean":"2024-09-13T09:07:34.288205+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:07:34.288205+0000","last_undegraded":"2024-09-13T09:07:34.288205+0000","last_fullsized":"2024-09-13T09:07:34.288205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"41'144","reported_seq":501,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.617472+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:07:40.617472+0000","last_peered":"2024-09-13T09:07:40.617472+0000","last_clean":"2024-09-13T09:07:40.617472+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:07:40.617472+0000","last_undegraded":"2024-09-13T09:07:40.617472+0000","last_fullsized":"2024-09-13T09:07:40.617472+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":2624315,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":981,"num_read_kb":2965,"num_write":241,"num_write_kb":602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"39'85","reported_seq":1039,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.309318+0000","last_change":"2024-09-13T09:07:15.873298+0000","last_active":"2024-09-13T09:07:39.309318+0000","last_peered":"2024-09-13T09:07:39.309318+0000","last_clean":"2024-09-13T09:07:39.309318+0000","last_became_active":"2024-09-13T09:07:13.852418+0000","last_became_peered":"2024-09-13T09:07:13.852418+0000","last_unstale":"2024-09-13T09:07:39.309318+0000","last_undegraded":"2024-09-13T09:07:39.309318+0000","last_fullsized":"2024-09-13T09:07:39.309318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:58.473601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":21401749,"num_objects":13,"num_object_clones":5,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":1532,"num_read_kb":6542,"num_write":96,"num_write_kb":617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":39,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.314425+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:07:39.314425+0000","last_peered":"2024-09-13T09:07:39.314425+0000","last_clean":"2024-09-13T09:07:39.314425+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:07:39.314425+0000","last_undegraded":"2024-09-13T09:07:39.314425+0000","last_fullsized":"2024-09-13T09:07:39.314425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"39'69","reported_seq":936,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-13T09:07:37.302333+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:07:37.302333+0000","last_peered":"2024-09-13T09:07:37.302333+0000","last_clean":"2024-09-13T09:07:37.302333+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:07:37.302333+0000","last_undegraded":"2024-09-13T09:07:37.302333+0000","last_fullsized":"2024-09-13T09:07:37.302333+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":9228532,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":884,"num_read_kb":3651,"num_write":65,"num_write_kb":3483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.306090+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:07:39.306090+0000","last_peered":"2024-09-13T09:07:39.306090+0000","last_clean":"2024-09-13T09:07:39.306090+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:07:39.306090+0000","last_undegraded":"2024-09-13T09:07:39.306090+0000","last_fullsized":"2024-09-13T09:07:39.306090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"35'35","reported_seq":179,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:07:38.305583+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:07:38.305583+0000","last_peered":"2024-09-13T09:07:38.305583+0000","last_clean":"2024-09-13T09:07:38.305583+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:07:38.305583+0000","last_undegraded":"2024-09-13T09:07:38.305583+0000","last_fullsized":"2024-09-13T09:07:38.305583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":3291686,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":135,"num_read_kb":3473,"num_write":30,"num_write_kb":1951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.314624+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:07:39.314624+0000","last_peered":"2024-09-13T09:07:39.314624+0000","last_clean":"2024-09-13T09:07:39.314624+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:07:39.314624+0000","last_undegraded":"2024-09-13T09:07:39.314624+0000","last_fullsized":"2024-09-13T09:07:39.314624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"41'87","reported_seq":233,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.193257+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:07:40.193257+0000","last_peered":"2024-09-13T09:07:40.193257+0000","last_clean":"2024-09-13T09:07:40.193257+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:07:40.193257+0000","last_undegraded":"2024-09-13T09:07:40.193257+0000","last_fullsized":"2024-09-13T09:07:40.193257+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":1048632,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":440,"num_read_kb":3253,"num_write":133,"num_write_kb":1280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.314562+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:07:39.314562+0000","last_peered":"2024-09-13T09:07:39.314562+0000","last_clean":"2024-09-13T09:07:39.314562+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:07:39.314562+0000","last_undegraded":"2024-09-13T09:07:39.314562+0000","last_fullsized":"2024-09-13T09:07:39.314562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T09:07:34.288309+0000","last_change":"2024-09-13T09:07:31.267483+0000","last_active":"2024-09-13T09:07:34.288309+0000","last_peered":"2024-09-13T09:07:34.288309+0000","last_clean":"2024-09-13T09:07:34.288309+0000","last_became_active":"2024-09-13T09:07:31.266543+0000","last_became_peered":"2024-09-13T09:07:31.266543+0000","last_unstale":"2024-09-13T09:07:34.288309+0000","last_undegraded":"2024-09-13T09:07:34.288309+0000","last_fullsized":"2024-09-13T09:07:34.288309+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:07:03.168839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.8","version":"41'48","reported_seq":1481,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.088406+0000","last_change":"2024-09-13T09:07:16.108021+0000","last_active":"2024-09-13T09:07:40.088406+0000","last_peered":"2024-09-13T09:07:40.088406+0000","last_clean":"2024-09-13T09:07:40.088406+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:07:40.088406+0000","last_undegraded":"2024-09-13T09:07:40.088406+0000","last_fullsized":"2024-09-13T09:07:40.088406+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":25508519,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1431,"num_read_kb":9749,"num_write":147,"num_write_kb":6682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"41'86","reported_seq":1551,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.621318+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:07:40.621318+0000","last_peered":"2024-09-13T09:07:40.621318+0000","last_clean":"2024-09-13T09:07:40.621318+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:07:40.621318+0000","last_undegraded":"2024-09-13T09:07:40.621318+0000","last_fullsized":"2024-09-13T09:07:40.621318+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":31528431,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":2056,"num_read_kb":6443,"num_write":86,"num_write_kb":3314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"42'80","reported_seq":439,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:41.146212+0000","last_change":"2024-09-13T09:07:28.553809+0000","last_active":"2024-09-13T09:07:41.146212+0000","last_peered":"2024-09-13T09:07:41.146212+0000","last_clean":"2024-09-13T09:07:41.146212+0000","last_became_active":"2024-09-13T09:07:28.521932+0000","last_became_peered":"2024-09-13T09:07:28.521932+0000","last_unstale":"2024-09-13T09:07:41.146212+0000","last_undegraded":"2024-09-13T09:07:41.146212+0000","last_fullsized":"2024-09-13T09:07:41.146212+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:45:52.270064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":23395607,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":354,"num_read_kb":3691,"num_write":93,"num_write_kb":7151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":836311,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"39'32","reported_seq":942,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:39.306188+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:07:39.306188+0000","last_peered":"2024-09-13T09:07:39.306188+0000","last_clean":"2024-09-13T09:07:39.306188+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:07:39.306188+0000","last_undegraded":"2024-09-13T09:07:39.306188+0000","last_fullsized":"2024-09-13T09:07:39.306188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":7723428,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":901,"num_read_kb":11661,"num_write":55,"num_write_kb":7368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"39'51","reported_seq":1103,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.618488+0000","last_change":"2024-09-13T09:07:15.873583+0000","last_active":"2024-09-13T09:07:40.618488+0000","last_peered":"2024-09-13T09:07:40.618488+0000","last_clean":"2024-09-13T09:07:40.618488+0000","last_became_active":"2024-09-13T09:07:13.859769+0000","last_became_peered":"2024-09-13T09:07:13.859769+0000","last_unstale":"2024-09-13T09:07:40.618488+0000","last_undegraded":"2024-09-13T09:07:40.618488+0000","last_fullsized":"2024-09-13T09:07:40.618488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:52.161129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":32366345,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":1043,"num_read_kb":20620,"num_write":99,"num_write_kb":12099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"39'35","reported_seq":1108,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.617709+0000","last_change":"2024-09-13T09:07:28.275191+0000","last_active":"2024-09-13T09:07:40.617709+0000","last_peered":"2024-09-13T09:07:40.617709+0000","last_clean":"2024-09-13T09:07:40.617709+0000","last_became_active":"2024-09-13T09:07:28.243825+0000","last_became_peered":"2024-09-13T09:07:28.243825+0000","last_unstale":"2024-09-13T09:07:40.617709+0000","last_undegraded":"2024-09-13T09:07:40.617709+0000","last_fullsized":"2024-09-13T09:07:40.617709+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:29:35.944961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":8698001,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":1018,"num_read_kb":5485,"num_write":31,"num_write_kb":420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":127486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"41'33","reported_seq":763,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.197241+0000","last_change":"2024-09-13T09:07:15.859429+0000","last_active":"2024-09-13T09:07:40.197241+0000","last_peered":"2024-09-13T09:07:40.197241+0000","last_clean":"2024-09-13T09:07:40.197241+0000","last_became_active":"2024-09-13T09:07:13.855153+0000","last_became_peered":"2024-09-13T09:07:13.855153+0000","last_unstale":"2024-09-13T09:07:40.197241+0000","last_undegraded":"2024-09-13T09:07:40.197241+0000","last_fullsized":"2024-09-13T09:07:40.197241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:15.261153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":14887392,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":703,"num_read_kb":8490,"num_write":30,"num_write_kb":1069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"41'53","reported_seq":681,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.620966+0000","last_change":"2024-09-13T09:07:28.284033+0000","last_active":"2024-09-13T09:07:40.620966+0000","last_peered":"2024-09-13T09:07:40.620966+0000","last_clean":"2024-09-13T09:07:40.620966+0000","last_became_active":"2024-09-13T09:07:28.238778+0000","last_became_peered":"2024-09-13T09:07:28.238778+0000","last_unstale":"2024-09-13T09:07:40.620966+0000","last_undegraded":"2024-09-13T09:07:40.620966+0000","last_fullsized":"2024-09-13T09:07:40.620966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:17:48.836969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":10573559,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":589,"num_read_kb":1423,"num_write":49,"num_write_kb":685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":51476,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"42'99","reported_seq":996,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:41.193418+0000","last_change":"2024-09-13T09:07:40.095409+0000","last_active":"2024-09-13T09:07:41.193418+0000","last_peered":"2024-09-13T09:07:41.193418+0000","last_clean":"2024-09-13T09:07:41.193418+0000","last_became_active":"2024-09-13T09:07:40.031971+0000","last_became_peered":"2024-09-13T09:07:40.031971+0000","last_unstale":"2024-09-13T09:07:41.193418+0000","last_undegraded":"2024-09-13T09:07:41.193418+0000","last_fullsized":"2024-09-13T09:07:41.193418+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:53:58.887694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":36122584,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":975,"num_read_kb":10314,"num_write":121,"num_write_kb":6972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":3809918,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"42'142","reported_seq":1190,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.617849+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:07:40.617849+0000","last_peered":"2024-09-13T09:07:40.617849+0000","last_clean":"2024-09-13T09:07:40.617849+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:07:40.617849+0000","last_undegraded":"2024-09-13T09:07:40.617849+0000","last_fullsized":"2024-09-13T09:07:40.617849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":5023237,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1760,"num_read_kb":3038,"num_write":208,"num_write_kb":598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"39'175","reported_seq":1600,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T09:07:38.570799+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:07:38.570799+0000","last_peered":"2024-09-13T09:07:38.570799+0000","last_clean":"2024-09-13T09:07:38.570799+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:07:38.570799+0000","last_undegraded":"2024-09-13T09:07:38.570799+0000","last_fullsized":"2024-09-13T09:07:38.570799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":26944762,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":2472,"num_read_kb":5928,"num_write":270,"num_write_kb":3531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"41'38","reported_seq":196,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T09:07:40.195232+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:07:40.195232+0000","last_peered":"2024-09-13T09:07:40.195232+0000","last_clean":"2024-09-13T09:07:40.195232+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:07:40.195232+0000","last_undegraded":"2024-09-13T09:07:40.195232+0000","last_fullsized":"2024-09-13T09:07:40.195232+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":2850922,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":192,"num_read_kb":2059,"num_write":45,"num_write_kb":3861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":486646571,"num_objects":402,"num_object_clones":118,"num_object_copies":804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":402,"num_whiteouts":127,"num_read":35566,"num_read_kb":242853,"num_write":3623,"num_write_kb":141262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":20831487,"num_keys_recovered":16,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204750848,"data_stored":218230603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2556,"ondisk_log_size":2556,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75288,"kb_used_data":40080,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94296552,"statfs":{"total":96636764160,"available":96559669248,"internally_reserved":0,"allocated":41041920,"data_stored":41500598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62332,"kb_used_data":31188,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94309508,"statfs":{"total":96636764160,"available":96572936192,"internally_reserved":0,"allocated":31936512,"data_stored":35153985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42748,"kb_used_data":11632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94329092,"statfs":{"total":96636764160,"available":96592990208,"internally_reserved":0,"allocated":11911168,"data_stored":12282400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]}]},{"osd":1,"up_from":0,"seq":64424509453,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66672,"kb_used_data":35556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94305168,"statfs":{"total":96636764160,"available":96568492032,"internally_reserved":0,"allocated":36409344,"data_stored":41632353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48492,"kb_used_data":13280,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94323348,"statfs":{"total":96636764160,"available":96587108352,"internally_reserved":0,"allocated":13598720,"data_stored":16880950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]}]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63160,"kb_used_data":27712,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94308680,"statfs":{"total":96636764160,"available":96572088320,"internally_reserved":0,"allocated":28377088,"data_stored":27955882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72116,"kb_used_data":36904,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94299724,"statfs":{"total":96636764160,"available":96562917376,"internally_reserved":0,"allocated":37789696,"data_stored":37387635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,66,75],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36769792,"data_stored":42246460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2920448,"data_stored":2862922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10850304,"data_stored":11494316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32641024,"data_stored":36120990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":27934720,"data_stored":27764400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":38076416,"data_stored":37927994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":14929920,"data_stored":18463082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40628224,"data_stored":41350439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:07:43.682 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:43.678+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:43.697 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:43.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:15159 3.c 3:33b101a6:::rbd_data.11242e26750a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15794176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:07:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.402103439999999, 2024-09-13T09:07:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052673589999999999, 2024-09-13T09:07:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 15159 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294458+0000", 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302503+0000", 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080450420000000005 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302534+0000", 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1352000000000001e-05 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302550+0000", 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5511000000000001e-05 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302617+0000", 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7722e-05 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302628+0000", 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0997e-05 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304649+0000", 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020203460000000001 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304734+0000", 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4839999999999994e-05 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.305520+0000", 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078672300000000002 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.343824+0000", 2024-09-13T09:07:43.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038304108000000003 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347049+0000", 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032249399999999999 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347102+0000", 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2163000000000002e-05 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347133+0000", 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1674999999999997e-05 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20975 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4112384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3443566459999996, 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052586704999999997, 2024-09-13T09:07:43.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20975 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352209+0000", 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.886e-06 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352216+0000", 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4159999999999998e-06 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352222+0000", 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404606+0000", 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052384209000000001 2024-09-13T09:07:43.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404695+0000", 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8455000000000006e-05 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404793+0000", 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8443999999999995e-05 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20976 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3443358809999992, 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052710422999999999, 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20976 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:43.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352230+0000", 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4829999999999999e-06 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352238+0000", 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.14e-06 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352245+0000", 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3130000000000004e-06 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404836+0000", 2024-09-13T09:07:43.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052590779999999997 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404881+0000", 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5034000000000003e-05 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404938+0000", 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7120999999999997e-05 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20977 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4227072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.344313262, 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052908033, 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20977 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:43.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352249+0000", 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352259+0000", 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9520000000000006e-06 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352268+0000", 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5550000000000003e-06 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352274+0000", 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404981+0000", 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052706483999999998 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405058+0000", 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7282999999999999e-05 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405158+0000", 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9699999999999998e-05 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20978 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3442566990000007, 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053008803, 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20978 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352305+0000", 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352312+0000", 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5339999999999996e-06 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352325+0000", 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2952000000000001e-05 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352333+0000", 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0180000000000001e-06 2024-09-13T09:07:43.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405203+0000", 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052870013 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405257+0000", 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4135000000000003e-05 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405315+0000", 2024-09-13T09:07:43.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8471999999999999e-05 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20979 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4308992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3442236560000005, 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053200718000000001, 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20979 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352342+0000", 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7610000000000001e-06 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352349+0000", 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352355+0000", 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405359+0000", 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053004355000000003 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405440+0000", 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0927999999999995e-05 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405540+0000", 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010015599999999999 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20980 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3442027240000005, 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053319995000000002, 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20980 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352362+0000", 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5959999999999998e-06 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352367+0000", 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4560000000000002e-06 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352370+0000", 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0860000000000002e-06 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405580+0000", 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053209583999999997 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405619+0000", 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9617999999999999e-05 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405680+0000", 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1259000000000004e-05 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20981 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4423680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3441906110000001, 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053531664999999999, 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20981 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:43.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352372+0000", 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352374+0000", 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5519999999999999e-06 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352377+0000", 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.258e-06 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352388+0000", 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1375e-05 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405720+0000", 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053331770000000001 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405800+0000", 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0050999999999996e-05 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405904+0000", 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103879 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20982 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4456448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3441718369999993, 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053719282, 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20982 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:43.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352393+0000", 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.615e-06 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352396+0000", 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6090000000000001e-06 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352399+0000", 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405931+0000", 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053532140999999998 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406011+0000", 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0226000000000001e-05 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406111+0000", 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9087999999999997e-05 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20983 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4505600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:43.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3439374599999994, 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053724453999999998, 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20983 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:43.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352401+0000", 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997754 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352631+0000", 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000230134 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352644+0000", 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2308e-05 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352651+0000", 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2030000000000003e-06 2024-09-13T09:07:43.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406162+0000", 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053510765000000002 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406241+0000", 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9764000000000004e-05 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406350+0000", 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010872199999999999 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20992 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4931584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.340713139, 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052559458000000003, 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20992 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355849+0000", 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355852+0000", 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2109999999999998e-06 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355859+0000", 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355864+0000", 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.028e-06 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408296+0000", 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052431658999999999 2024-09-13T09:07:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408350+0000", 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4253999999999999e-05 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408409+0000", 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9247999999999999e-05 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20993 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5013504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3406734329999992, 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052710119999999999, 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20993 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355889+0000", 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355893+0000", 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2509999999999996e-06 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355903+0000", 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355909+0000", 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0190000000000003e-06 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408441+0000", 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052532395000000003 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408515+0000", 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4090000000000004e-05 2024-09-13T09:07:43.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408600+0000", 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4716999999999995e-05 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20994 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5046272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:43.756 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3406490570000003, 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052842215999999997, 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20994 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:43.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355913+0000", 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355916+0000", 2024-09-13T09:07:43.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355923+0000", 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355931+0000", 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5280000000000003e-06 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408630+0000", 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052698997999999997 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408698+0000", 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7966999999999995e-05 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408756+0000", 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8233000000000003e-05 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20995 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5095424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3406260099999994, 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052955222000000003, 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20995 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:43.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355940+0000", 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1669999999999999e-06 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355946+0000", 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355951+0000", 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9309999999999999e-06 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408783+0000", 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052831751000000003 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408838+0000", 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5031000000000002e-05 2024-09-13T09:07:43.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408892+0000", 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4134e-05 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20996 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5128192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3406070159999999, 2024-09-13T09:07:43.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053075004000000002, 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20996 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355959+0000", 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:07:43.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355965+0000", 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355972+0000", 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408922+0000", 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052950174000000003 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408973+0000", 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0837000000000001e-05 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409031+0000", 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8455000000000002e-05 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20997 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5210112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3405869680000002, 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053241038999999997, 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20997 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355979+0000", 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0170000000000001e-06 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355985+0000", 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.122e-06 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355990+0000", 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409061+0000", 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053071209000000001 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409127+0000", 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5704999999999995e-05 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409217+0000", 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0492e-05 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20998 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5242880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3405685609999995, 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053395182999999999, 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20998 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355994+0000", 2024-09-13T09:07:43.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355997+0000", 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7470000000000001e-06 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356003+0000", 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356009+0000", 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1990000000000003e-06 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409244+0000", 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053234721999999998 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409321+0000", 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6998999999999999e-05 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409390+0000", 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8943999999999996e-05 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20999 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5292032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3405495050000003, 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053549106999999999, 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20999 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356013+0000", 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356016+0000", 2024-09-13T09:07:43.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.016e-06 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356023+0000", 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3810000000000004e-06 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356038+0000", 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5305999999999999e-05 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409408+0000", 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053369650999999997 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409472+0000", 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4587999999999999e-05 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409563+0000", 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0566e-05 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:21000 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3405185310000007, 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053705747999999998, 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21000 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356044+0000", 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356048+0000", 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.985e-06 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356056+0000", 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356061+0000", 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409590+0000", 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053528986000000001 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409663+0000", 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3342000000000004e-05 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409750+0000", 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7349999999999995e-05 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:21001 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.3404951769999993, 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053808105000000002, 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:43.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21001 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:43.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356067+0000", 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356080+0000", 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2228999999999999e-05 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356088+0000", 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356093+0000", 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409777+0000", 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053683869000000002 2024-09-13T09:07:43.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409824+0000", 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6814000000000003e-05 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409876+0000", 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2405999999999999e-05 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:43.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:43.793 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:43.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:43.795 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:43.790+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:43.895 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:43.892+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:43.977 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:43.977 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:43.977 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:43.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:43.996 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:43.994+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:44.068 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-13T09:07:44.097 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:44.092+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:44.153 INFO:teuthology.orchestra.run.smithi031.stdout:1387 read 0x639c5b2 thru 0x63a0552 (0x3fa1 bytes) 2024-09-13T09:07:44.155 INFO:teuthology.orchestra.run.smithi031.stdout:1391 punch from 0x5c95b6c to 0x5ca48ed, (0xed81 bytes) 2024-09-13T09:07:44.163 INFO:teuthology.orchestra.run.smithi031.stdout:1392 punch from 0xa28982d to 0xa29825d, (0xea30 bytes) 2024-09-13T09:07:44.163 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:44.163 INFO:teuthology.orchestra.run.smithi031.stdout:1395 writesame 0xe210352 thru 0xe21db52 data_size 0xb(0xd801 bytes) 2024-09-13T09:07:44.179 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:44.179 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:44.180 INFO:teuthology.orchestra.run.smithi031.stdout:1396 punch from 0x68eca0 to 0x6910f9, (0x2459 bytes) 2024-09-13T09:07:44.180 INFO:teuthology.orchestra.run.smithi031.stdout:1399 read 0xc0a845a thru 0xc0adfd4 (0x5b7b bytes) 2024-09-13T09:07:44.181 INFO:teuthology.orchestra.run.smithi031.stdout:1400 write 0x945eada thru 0x9461ba6 (0x30cd bytes) 2024-09-13T09:07:44.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:44.226 INFO:teuthology.orchestra.run.smithi031.stdout:1401 trunc from 0xe45870c to 0x6538c1 2024-09-13T09:07:44.227 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:44.224+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:44.299 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:44.294+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:44.356 INFO:teuthology.orchestra.run.smithi031.stdout:1402 write 0x54e7004 thru 0x54f1e3b (0xae38 bytes) 2024-09-13T09:07:44.372 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:44.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.674359588999998, 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840076, 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 34 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:44.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694022+0000", 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694033+0000", 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.116e-05 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694051+0000", 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694065+0000", 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3716e-05 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700273+0000", 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062082320000000002 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700405+0000", 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131416 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700608+0000", 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020368099999999999 2024-09-13T09:07:44.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705556+0000", 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049475760000000004 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705654+0000", 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8444999999999997e-05 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705741+0000", 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6416999999999996e-05 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705863+0000", 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121876 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.378 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.674291752000002, 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:44.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.673983737999997, 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:44.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.673895825000002, 2024-09-13T09:07:44.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:44.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:44.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:44.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.403371464000003, 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:44.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.403283700999999, 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:44.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:44.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:44.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.403231648000002, 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:44.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:44.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:44.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.403046566, 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:44.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:44.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.402972077000001, 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.424 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:44.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:44.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.402917756999997, 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:44.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:44.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:44.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.402719672000003, 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:44.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.402638408999998, 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:44.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:44.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.908343694000003, 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:44.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.908028057000003, 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:44.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:44.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.907967726999999, 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:44.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:44.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:44.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.907909820999997, 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:44.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:44.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.907607224000003, 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:44.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:44.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7649 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:44.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.515042056999999, 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013081786, 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7649 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:44.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853339+0000", 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853345+0000", 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853365+0000", 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9054000000000001e-05 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853374+0000", 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858418+0000", 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050442869999999997 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858525+0000", 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010602099999999999 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858992+0000", 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046758299999999999 2024-09-13T09:07:44.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.861712+0000", 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027195700000000001 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866315+0000", 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046037439999999999 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866381+0000", 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5121000000000002e-05 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866422+0000", 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1331000000000001e-05 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7657 3.10 3:08eae27d:::rbd_data.11242ae5e73a.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13762560~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:44.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.500249762999999, 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018927840000000001, 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7657 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868131+0000", 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870832+0000", 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027013660000000002 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870853+0000", 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.107e-05 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870862+0000", 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7830000000000004e-06 2024-09-13T09:07:44.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870910+0000", 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871083+0000", 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017277500000000001 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871386+0000", 2024-09-13T09:07:44.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030378600000000001 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878951+0000", 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075642649999999997 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886982+0000", 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080316160000000001 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887025+0000", 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3288999999999998e-05 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887060+0000", 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4706999999999998e-05 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7660 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.497506674, 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016257230000000001, 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7660 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:44.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870874+0000", 2024-09-13T09:07:44.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870879+0000", 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870885+0000", 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870888+0000", 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876842+0000", 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059541780000000001 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876952+0000", 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109577 2024-09-13T09:07:44.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.877351+0000", 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039971899999999999 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878995+0000", 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001643179 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:44.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887073+0000", 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080780109999999995 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887100+0000", 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7585000000000001e-05 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887133+0000", 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2533000000000002e-05 2024-09-13T09:07:44.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:44.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:44.500 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:1405 writesame 0xdfaf0a2 thru 0xdfaf0d8 data_size 0x1(0x37 bytes) 2024-09-13T09:07:44.501 INFO:teuthology.orchestra.run.smithi031.stdout:1406 write 0xb621677 thru 0xb625e48 (0x47d2 bytes) 2024-09-13T09:07:44.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:44.503 INFO:teuthology.orchestra.run.smithi031.stdout:1407 writesame 0x90f43bf thru 0x90f7b82 data_size 0x4(0x37c4 bytes) 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:1410 punch from 0x9d428de to 0x9d50405, (0xdb27 bytes) 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:1413 writesame 0xd54d74f thru 0xd556ae9 data_size 0x1(0x939b bytes) 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:1414 read 0x883abb1 thru 0x884473c (0x9b8c bytes) 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:1416 punch from 0xd1d951f to 0xd1e0b93, (0x7674 bytes) 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:1417 writesame 0xa1d4219 thru 0xa1e0a73 data_size 0x7b(0xc85b bytes) 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:1421 read 0x2270500 thru 0x227ac66 (0xa767 bytes) 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:1422 write 0x4078f05 thru 0x407cd9b (0x3e97 bytes) 2024-09-13T09:07:44.504 INFO:teuthology.orchestra.run.smithi031.stdout:1423 trunc from 0xdfaf0d9 to 0x8138764 2024-09-13T09:07:44.504 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:44.496+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:44.506 INFO:teuthology.orchestra.run.smithi031.stdout:1424 write 0x5b19755 thru 0x5b1eb2f (0x53db bytes) 2024-09-13T09:07:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:44.515 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:1434 punch from 0xc6f101 to 0xc74fe8, (0x5ee7 bytes) 2024-09-13T09:07:44.516 INFO:teuthology.orchestra.run.smithi031.stdout:1435 clone 13 order 19 su 8192 sc 8 2024-09-13T09:07:44.603 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:44.598+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:44.634 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:44.635 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:44.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:44.738 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:44.736+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:44.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:07:44.804 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:44.802+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:44.825 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:44.825 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:44.825 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:44.825 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:44.826 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:44.843 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:44.892 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone12 2024-09-13T09:07:44.905 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:44.902+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:45.005 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:45.000+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6816993099999991, 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:07:45.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:07:45.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6816825370000004, 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:07:45.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6816452319999993, 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.035 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6816255180000006, 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:07:45.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6816091929999999, 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:45.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:07:45.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6815933269999999, 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:07:45.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:07:45.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:45.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6815775419999994, 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:45.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:07:45.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6815584710000007, 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:07:45.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6815419239999994, 2024-09-13T09:07:45.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:07:45.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6815143250000002, 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:07:45.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:07:45.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.681497083, 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:45.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:07:45.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.68146664, 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:45.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6814477249999999, 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:07:45.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:07:45.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6814299370000008, 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:07:45.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:07:45.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6814047129999992, 2024-09-13T09:07:45.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:07:45.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6813865089999993, 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:07:45.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6813661910000004, 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:07:45.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:07:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6813476430000005, 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:07:45.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:07:45.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:07:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6813303689999994, 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6813129149999995, 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:07:45.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:07:45.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:07:45.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.117 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:45.118 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-13T09:07:45.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:45.119 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:45.114+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:45.219 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:45.214+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:45.284 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:45.284 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:45.284 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:45.284 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:45.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:45.346 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:45.344+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:45.421 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:45.418+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:45.423 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:45.423 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:45.423 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:45.423 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:45.423 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:45.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:45.491 INFO:teuthology.orchestra.run.smithi031.stdout:1436 writesame 0x3d503d5 thru 0x3d5f4d4 data_size 0x40(0xf100 bytes) 2024-09-13T09:07:45.516 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:45.516 INFO:teuthology.orchestra.run.smithi031.stdout:1440 write 0x73deb0c thru 0x73ee8fa (0xfdef bytes) 2024-09-13T09:07:45.521 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:45.518+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:45.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:45.541 INFO:teuthology.orchestra.run.smithi031.stdout:1441 trunc from 0x8138764 to 0x1ea3621 2024-09-13T09:07:45.577 INFO:teuthology.orchestra.run.smithi031.stdout:1442 punch from 0x1088ce7 to 0x108b1aa, (0x24c3 bytes) 2024-09-13T09:07:45.607 INFO:teuthology.orchestra.run.smithi031.stdout:1444 write 0xe93dd76 thru 0xe946e8e (0x9119 bytes) 2024-09-13T09:07:45.622 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:45.618+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:45.623 INFO:teuthology.orchestra.run.smithi031.stdout:1445 write 0x7fcc1a6 thru 0x7fd39d3 (0x782e bytes) 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.500619174000001, 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:45.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:07:45.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:07:45.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:45.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:07:45.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14831 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:45.640 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.394811161, 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071499264000000007, 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14831 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236541+0000", 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3910000000000001e-06 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236548+0000", 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236553+0000", 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307720+0000", 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071166574999999996 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307883+0000", 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000163809 2024-09-13T09:07:45.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308039+0000", 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155229 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14834 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393488494, 2024-09-13T09:07:45.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070919408000000003, 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14834 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237864+0000", 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237871+0000", 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237877+0000", 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:07:45.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308614+0000", 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070737169000000003 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308704+0000", 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0357000000000002e-05 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308782+0000", 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7480999999999999e-05 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14835 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393453269, 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071108525000000006, 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14835 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237908+0000", 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1191e-05 2024-09-13T09:07:45.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237929+0000", 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1189e-05 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237936+0000", 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308827+0000", 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070891320999999993 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308924+0000", 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6665999999999999e-05 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309006+0000", 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2231000000000006e-05 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14836 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393408292, 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071290567999999999, 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14836 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237945+0000", 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6229999999999999e-06 2024-09-13T09:07:45.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237953+0000", 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237958+0000", 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0309999999999998e-06 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309041+0000", 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071083576999999995 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309132+0000", 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0500000000000004e-05 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309233+0000", 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101116 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14837 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393388485999999, 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071483336999999994, 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14837 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237964+0000", 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237971+0000", 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:07:45.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237975+0000", 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6360000000000002e-06 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309280+0000", 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071304765000000006 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309359+0000", 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9066999999999994e-05 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309445+0000", 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6255999999999995e-05 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14838 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393368896, 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071691981000000002, 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14838 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237981+0000", 2024-09-13T09:07:45.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237984+0000", 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237990+0000", 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237995+0000", 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309487+0000", 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071492752000000007 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309575+0000", 2024-09-13T09:07:45.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7106000000000003e-05 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309674+0000", 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9208000000000003e-05 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14844 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:45.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393351463, 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072094264000000005, 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14844 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238002+0000", 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238008+0000", 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.032e-06 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238013+0000", 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309739+0000", 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071726390000000001 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309786+0000", 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6561999999999999e-05 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310093+0000", 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030766000000000002 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14845 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393333302, 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072306747000000005, 2024-09-13T09:07:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14845 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238019+0000", 2024-09-13T09:07:45.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238025+0000", 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4870000000000002e-06 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238029+0000", 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3769999999999998e-06 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310139+0000", 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072110056000000006 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310224+0000", 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4291999999999999e-05 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310324+0000", 2024-09-13T09:07:45.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100405 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14846 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1343488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393315538, 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072637322000000004, 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14846 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238037+0000", 2024-09-13T09:07:45.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238045+0000", 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0960000000000001e-06 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238051+0000", 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310365+0000", 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072313575000000005 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310432+0000", 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6888000000000001e-05 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310672+0000", 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024050699999999999 2024-09-13T09:07:45.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14847 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393294407000001, 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072846102999999995, 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14847 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:45.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238058+0000", 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5119999999999998e-06 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238063+0000", 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238067+0000", 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4750000000000004e-06 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310717+0000", 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072649243000000002 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310803+0000", 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6147e-05 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310902+0000", 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9474000000000003e-05 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14848 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.393278776000001, 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072985396999999994, 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14848 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238074+0000", 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.227e-06 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238080+0000", 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238085+0000", 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1410000000000004e-06 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310939+0000", 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072853862000000005 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310997+0000", 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8248000000000002e-05 2024-09-13T09:07:45.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311057+0000", 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0071999999999997e-05 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14849 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.392344281, 2024-09-13T09:07:45.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072287373000000002, 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14849 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238089+0000", 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9990826 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239012+0000", 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092334199999999996 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239026+0000", 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3351e-05 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239034+0000", 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4419999999999998e-06 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311104+0000", 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072069464 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311194+0000", 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9922999999999999e-05 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311294+0000", 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001002 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14850 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3702784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.39230832, 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072393710999999999, 2024-09-13T09:07:45.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14850 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239045+0000", 2024-09-13T09:07:45.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4429999999999999e-06 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239053+0000", 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5179999999999998e-06 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239058+0000", 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311330+0000", 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072271613999999998 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311381+0000", 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0908000000000001e-05 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311436+0000", 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5513000000000002e-05 2024-09-13T09:07:45.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14861 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.392085373, 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072582646000000001, 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14861 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239270+0000", 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239280+0000", 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0210999999999999e-05 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239285+0000", 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.967e-06 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311481+0000", 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072196249000000004 2024-09-13T09:07:45.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311573+0000", 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1588999999999993e-05 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311848+0000", 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027526699999999999 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14862 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.392062063000001, 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072869149999999994, 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14862 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239288+0000", 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239295+0000", 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239302+0000", 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.793e-06 2024-09-13T09:07:45.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239305+0000", 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9380000000000001e-06 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311878+0000", 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072573031999999996 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311923+0000", 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4351000000000003e-05 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312158+0000", 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023512399999999999 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14863 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.392042799, 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072990372999999997, 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14863 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239309+0000", 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939999999999999e-06 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239312+0000", 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239315+0000", 2024-09-13T09:07:45.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7630000000000001e-06 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312186+0000", 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072871378000000001 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312243+0000", 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6487000000000003e-05 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312298+0000", 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5506e-05 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14864 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.391982989000001, 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073255022000000003, 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14864 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239317+0000", 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999949 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239372+0000", 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5457000000000001e-05 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239381+0000", 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.003e-06 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239387+0000", 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6130000000000003e-06 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312327+0000", 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072940489999999997 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312409+0000", 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.153e-05 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312623+0000", 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213799 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14865 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.391712464999999, 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073307136999999994, 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14865 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239390+0000", 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999752 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239643+0000", 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025296200000000002 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239653+0000", 2024-09-13T09:07:45.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7839999999999998e-06 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239660+0000", 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0570000000000004e-06 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312655+0000", 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072994857999999996 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312731+0000", 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5754999999999996e-05 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312945+0000", 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021457799999999999 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14866 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:45.726 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.391684294999999, 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073614097000000003, 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14866 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:45.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239669+0000", 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.858e-06 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239675+0000", 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239680+0000", 2024-09-13T09:07:45.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312975+0000", 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073294697000000006 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313051+0000", 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6297999999999994e-05 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:45.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:45.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313280+0000", 2024-09-13T09:07:45.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022920800000000001 2024-09-13T09:07:45.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:45.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:45.731 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:45.733 INFO:teuthology.orchestra.run.smithi031.stdout:1446 read 0xb66df46 thru 0xb6712fe (0x33b9 bytes) 2024-09-13T09:07:45.733 INFO:teuthology.orchestra.run.smithi031.stdout:1448 writesame 0xa5a1bb3 thru 0xa5ab544 data_size 0xb(0x9992 bytes) 2024-09-13T09:07:45.733 INFO:teuthology.orchestra.run.smithi031.stdout:1449 read 0x34b4c11 thru 0x34c3700 (0xeaf0 bytes) 2024-09-13T09:07:45.733 INFO:teuthology.orchestra.run.smithi031.stdout:1450 read 0xabc45eb thru 0xabc872e (0x4144 bytes) 2024-09-13T09:07:45.733 INFO:teuthology.orchestra.run.smithi031.stdout:1451 punch from 0x3fd343e to 0x3fd5803, (0x23c5 bytes) 2024-09-13T09:07:45.733 INFO:teuthology.orchestra.run.smithi031.stdout:1452 read 0xbf3955d thru 0xbf3aae5 (0x1589 bytes) 2024-09-13T09:07:45.733 INFO:teuthology.orchestra.run.smithi031.stdout:1453 write 0xdf79bcd thru 0xdf80124 (0x6558 bytes) 2024-09-13T09:07:45.734 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:45.735 INFO:teuthology.orchestra.run.smithi031.stdout:1454 write 0xc1994b4 thru 0xc19e1f3 (0x4d40 bytes) 2024-09-13T09:07:45.735 INFO:teuthology.orchestra.run.smithi031.stdout:1456 read 0xdf5591b thru 0xdf5f71b (0x9e01 bytes) 2024-09-13T09:07:45.735 INFO:teuthology.orchestra.run.smithi031.stdout:1457 punch from 0x2e33226 to 0x2e3753a, (0x4314 bytes) 2024-09-13T09:07:45.735 INFO:teuthology.orchestra.run.smithi031.stdout:1459 write 0x728b77f thru 0x7290e64 (0x56e6 bytes) 2024-09-13T09:07:45.735 INFO:teuthology.orchestra.run.smithi031.stdout:1460 trunc from 0xe946e8f to 0xda2786f 2024-09-13T09:07:45.735 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:45.736 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:45.736 INFO:teuthology.orchestra.run.smithi031.stdout:1461 write 0x22dffc4 thru 0x22e0ecc (0xf09 bytes) 2024-09-13T09:07:45.737 INFO:teuthology.orchestra.run.smithi031.stdout:1462 read 0x25f13f4 thru 0x25ffbea (0xe7f7 bytes) 2024-09-13T09:07:45.737 INFO:teuthology.orchestra.run.smithi031.stdout:1463 read 0x3f7f92 thru 0x406e85 (0xeef4 bytes) 2024-09-13T09:07:45.737 INFO:teuthology.orchestra.run.smithi031.stdout:1465 punch from 0xaf850d6 to 0xaf8909f, (0x3fc9 bytes) 2024-09-13T09:07:45.737 INFO:teuthology.orchestra.run.smithi031.stdout:1466 writesame 0x3ee0deb thru 0x3eef6f6 data_size 0x26(0xe90c bytes) 2024-09-13T09:07:45.737 INFO:teuthology.orchestra.run.smithi031.stdout:1468 read 0xa99fbd5 thru 0xa9a5e32 (0x625e bytes) 2024-09-13T09:07:45.737 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:45.737 INFO:teuthology.orchestra.run.smithi031.stdout:1470 clone 14 order 21 su 1048576 sc 3 2024-09-13T09:07:45.737 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:45.738 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:45.738 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:45.738 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:45.739 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:45.730+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:45.751 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:45.751 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:45.771 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:45.771 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:45.790 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:45.794 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:07:45.810 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:45.833 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:45.833 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:45.833 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:45.828+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:45.876 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone13 from 0xda2786f (overlap 0x1ea3621) to 0x1b930b7 2024-09-13T09:07:45.881 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:45.881 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:45.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:45.934 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:45.932+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:45.975 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-13T09:07:46.026 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:46.026 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:46.035 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:46.032+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:46.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:46.135 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:46.130+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:46.164 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:46.164 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:46.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:46.268 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:46.266+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:46.278 INFO:teuthology.orchestra.run.smithi031.stdout:1472 write 0x522152e thru 0x5227712 (0x61e5 bytes) 2024-09-13T09:07:46.289 INFO:teuthology.orchestra.run.smithi031.stdout:1473 read 0xace7818 thru 0xacf7209 (0xf9f2 bytes) 2024-09-13T09:07:46.289 INFO:teuthology.orchestra.run.smithi031.stdout:1475 punch from 0xce800ea to 0xce8c19b, (0xc0b1 bytes) 2024-09-13T09:07:46.291 INFO:teuthology.orchestra.run.smithi031.stdout:1477 trunc from 0xda2786f to 0xcc8ec7d 2024-09-13T09:07:46.306 INFO:teuthology.orchestra.run.smithi031.stdout:1478 write 0x2731a1f thru 0x2737994 (0x5f76 bytes) 2024-09-13T09:07:46.314 INFO:teuthology.orchestra.run.smithi031.stdout:1479 read 0x63f13b5 thru 0x63f1ebe (0xb0a bytes) 2024-09-13T09:07:46.314 INFO:teuthology.orchestra.run.smithi031.stdout:1480 writesame 0x69e830c thru 0x69f5a24 data_size 0x5(0xd719 bytes) 2024-09-13T09:07:46.334 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:46.334 INFO:teuthology.orchestra.run.smithi031.stdout:1483 read 0x51b7235 thru 0x51c6443 (0xf20f bytes) 2024-09-13T09:07:46.335 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:46.335 INFO:teuthology.orchestra.run.smithi031.stdout:1486 read 0x22ff018 thru 0x22fff16 (0xeff bytes) 2024-09-13T09:07:46.336 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:46.332+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:46.337 INFO:teuthology.orchestra.run.smithi031.stdout:1487 write 0xb9ad379 thru 0xb9b2874 (0x54fc bytes) 2024-09-13T09:07:46.343 INFO:teuthology.orchestra.run.smithi031.stdout:1488 write 0xb3961aa thru 0xb39a4b8 (0x430f bytes) 2024-09-13T09:07:46.351 INFO:teuthology.orchestra.run.smithi031.stdout:1490 read 0x2f766e9 thru 0x2f81f7b (0xb893 bytes) 2024-09-13T09:07:46.351 INFO:teuthology.orchestra.run.smithi031.stdout:1493 read 0xad57eee thru 0xad5b5f4 (0x3707 bytes) 2024-09-13T09:07:46.353 INFO:teuthology.orchestra.run.smithi031.stdout:1494 trunc from 0xcc8ec7d to 0xc26057e 2024-09-13T09:07:46.358 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:46.358 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:46.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:46.389 INFO:teuthology.orchestra.run.smithi031.stdout:1496 writesame 0x4b68474 thru 0x4b72534 data_size 0x1(0xa0c1 bytes) 2024-09-13T09:07:46.455 INFO:teuthology.orchestra.run.smithi031.stdout:1497 write 0xaaadd32 thru 0xaab9cab (0xbf7a bytes) 2024-09-13T09:07:46.465 INFO:teuthology.orchestra.run.smithi031.stdout:1498 read 0x5eb287c thru 0x5ebd090 (0xa815 bytes) 2024-09-13T09:07:46.468 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:46.466+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:46.483 INFO:teuthology.orchestra.run.smithi031.stdout:1499 trunc from 0xc26057e to 0x5934fae 2024-09-13T09:07:46.513 INFO:teuthology.orchestra.run.smithi031.stdout:1500 punch from 0x194a134 to 0x194b92a, (0x17f6 bytes) 2024-09-13T09:07:46.531 INFO:teuthology.orchestra.run.smithi031.stdout:1501 trunc from 0x5934fae to 0xc949424 2024-09-13T09:07:46.538 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:46.532+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:46.543 INFO:teuthology.orchestra.run.smithi031.stdout:1502 punch from 0x3de9bc5 to 0x3dedd17, (0x4152 bytes) 2024-09-13T09:07:46.543 INFO:teuthology.orchestra.run.smithi031.stdout:1503 write 0xe78b2b thru 0xe85161 (0xc637 bytes) 2024-09-13T09:07:46.552 INFO:teuthology.orchestra.run.smithi031.stdout:1505 write 0x8935f45 thru 0x893c642 (0x66fe bytes) 2024-09-13T09:07:46.557 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:46.557 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:46.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:46.575 INFO:teuthology.orchestra.run.smithi031.stdout:1506 writesame 0xd75928f thru 0xd759a40 data_size 0x5(0x7b2 bytes) 2024-09-13T09:07:46.592 INFO:teuthology.orchestra.run.smithi031.stdout:1507 writesame 0xe817898 thru 0xe822ef0 data_size 0x1(0xb659 bytes) 2024-09-13T09:07:46.592 INFO:teuthology.orchestra.run.smithi031.stdout:1508 write 0x227dad7 thru 0x2283e4b (0x6375 bytes) 2024-09-13T09:07:46.598 INFO:teuthology.orchestra.run.smithi031.stdout:1509 write 0x7039033 thru 0x703d921 (0x48ef bytes) 2024-09-13T09:07:46.603 INFO:teuthology.orchestra.run.smithi031.stdout:1510 read 0xc43a945 thru 0xc4447f2 (0x9eae bytes) 2024-09-13T09:07:46.603 INFO:teuthology.orchestra.run.smithi031.stdout:1511 punch from 0xba88cee to 0xba95382, (0xc694 bytes) 2024-09-13T09:07:46.603 INFO:teuthology.orchestra.run.smithi031.stdout:1512 write 0x957e6e9 thru 0x95870b6 (0x89ce bytes) 2024-09-13T09:07:46.608 INFO:teuthology.orchestra.run.smithi031.stdout:1515 punch from 0x9e79c73 to 0x9e8061d, (0x69aa bytes) 2024-09-13T09:07:46.609 INFO:teuthology.orchestra.run.smithi031.stdout:1516 write 0x6dbabd3 thru 0x6dc92bd (0xe6eb bytes) 2024-09-13T09:07:46.611 INFO:teuthology.orchestra.run.smithi031.stdout:1517 read 0xbe058d4 thru 0xbe06af2 (0x121f bytes) 2024-09-13T09:07:46.611 INFO:teuthology.orchestra.run.smithi031.stdout:1518 flatten 2024-09-13T09:07:46.665 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:46.662+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:46.729 INFO:teuthology.orchestra.run.smithi031.stdout:1519 punch from 0x42099cf to 0x421422e, (0xa85f bytes) 2024-09-13T09:07:46.729 INFO:teuthology.orchestra.run.smithi031.stdout:1520 write 0x6764fe3 thru 0x6770146 (0xb164 bytes) 2024-09-13T09:07:46.732 INFO:teuthology.orchestra.run.smithi031.stdout:1521 read 0x66e8197 thru 0x66f4093 (0xbefd bytes) 2024-09-13T09:07:46.733 INFO:teuthology.orchestra.run.smithi031.stdout:1522 punch from 0x9a5daac to 0x9a6572e, (0x7c82 bytes) 2024-09-13T09:07:46.740 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:46.736+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:46.755 INFO:teuthology.orchestra.run.smithi031.stdout:1523 trunc from 0xe822ef1 to 0xfab1b2 2024-09-13T09:07:46.762 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:46.762 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:46.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:46.781 INFO:teuthology.orchestra.run.smithi031.stdout:1524 write 0x411c3cf thru 0x4123764 (0x7396 bytes) 2024-09-13T09:07:46.790 INFO:teuthology.orchestra.run.smithi031.stdout:1525 trunc from 0x4123765 to 0x74b7b45 2024-09-13T09:07:46.797 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:46.797 INFO:teuthology.orchestra.run.smithi031.stdout:1527 punch from 0x71a8346 to 0x71b0a73, (0x872d bytes) 2024-09-13T09:07:46.797 INFO:teuthology.orchestra.run.smithi031.stdout:1529 read 0x25f994a thru 0x260597b (0xc032 bytes) 2024-09-13T09:07:46.797 INFO:teuthology.orchestra.run.smithi031.stdout:1530 write 0x130b967 thru 0x130eed0 (0x356a bytes) 2024-09-13T09:07:46.801 INFO:teuthology.orchestra.run.smithi031.stdout:1532 punch from 0x4f7251b to 0x4f7b624, (0x9109 bytes) 2024-09-13T09:07:46.801 INFO:teuthology.orchestra.run.smithi031.stdout:1533 punch from 0x3afe718 to 0x3b05c38, (0x7520 bytes) 2024-09-13T09:07:46.801 INFO:teuthology.orchestra.run.smithi031.stdout:1534 write 0x43299b5 thru 0x4329afe (0x14a bytes) 2024-09-13T09:07:46.818 INFO:teuthology.orchestra.run.smithi031.stdout:1535 write 0xa59f385 thru 0xa5a8955 (0x95d1 bytes) 2024-09-13T09:07:46.822 INFO:teuthology.orchestra.run.smithi031.stdout:1536 read 0x2ef9170 thru 0x2f03356 (0xa1e7 bytes) 2024-09-13T09:07:46.840 INFO:teuthology.orchestra.run.smithi031.stdout:1537 trunc from 0xa5a8956 to 0x2c8e2e 2024-09-13T09:07:46.840 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:46.836+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:46.856 INFO:teuthology.orchestra.run.smithi031.stdout:1538 write 0x14cc17 thru 0x157269 (0xa653 bytes) 2024-09-13T09:07:46.869 INFO:teuthology.orchestra.run.smithi031.stdout:1540 writesame 0x1643b61 thru 0x1653492 data_size 0x2(0xf932 bytes) 2024-09-13T09:07:46.891 INFO:teuthology.orchestra.run.smithi031.stdout:1544 write 0x7c5c767 thru 0x7c631b4 (0x6a4e bytes) 2024-09-13T09:07:46.915 INFO:teuthology.orchestra.run.smithi031.stdout:1545 write 0xab9f562 thru 0xaba88a3 (0x9342 bytes) 2024-09-13T09:07:46.921 INFO:teuthology.orchestra.run.smithi031.stdout:1546 punch from 0x3b1f1fb to 0x3b2a912, (0xb717 bytes) 2024-09-13T09:07:46.922 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:46.922 INFO:teuthology.orchestra.run.smithi031.stdout:1548 read 0x97d392a thru 0x97e0245 (0xc91c bytes) 2024-09-13T09:07:46.922 INFO:teuthology.orchestra.run.smithi031.stdout:1549 read 0x808254e thru 0x808e988 (0xc43b bytes) 2024-09-13T09:07:46.930 INFO:teuthology.orchestra.run.smithi031.stdout:1551 trunc from 0xaba88a4 to 0xe7d66f8 2024-09-13T09:07:46.940 INFO:teuthology.orchestra.run.smithi031.stdout:1555 write 0x784e0bd thru 0x7851e37 (0x3d7b bytes) 2024-09-13T09:07:46.941 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:46.936+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:46.943 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:46.943 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:46.946 INFO:teuthology.orchestra.run.smithi031.stdout:1556 punch from 0x89f9da to 0x8adb83, (0xe1a9 bytes) 2024-09-13T09:07:46.947 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:46.947 INFO:teuthology.orchestra.run.smithi031.stdout:1558 read 0x786b3a9 thru 0x787131a (0x5f72 bytes) 2024-09-13T09:07:46.947 INFO:teuthology.orchestra.run.smithi031.stdout:1560 writesame 0x70ea57e thru 0x70ee177 data_size 0x9(0x3bfa bytes) 2024-09-13T09:07:46.956 INFO:teuthology.orchestra.run.smithi031.stdout:1561 write 0xcc3c9a1 thru 0xcc3d6bc (0xd1c bytes) 2024-09-13T09:07:46.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:46.963 INFO:teuthology.orchestra.run.smithi031.stdout:1562 write 0x81fcc7a thru 0x820947e (0xc805 bytes) 2024-09-13T09:07:46.969 INFO:teuthology.orchestra.run.smithi031.stdout:1563 read 0xc54c29a thru 0xc557849 (0xb5b0 bytes) 2024-09-13T09:07:46.969 INFO:teuthology.orchestra.run.smithi031.stdout:1564 read 0x85cf863 thru 0x85d56b6 (0x5e54 bytes) 2024-09-13T09:07:46.969 INFO:teuthology.orchestra.run.smithi031.stdout:1565 read 0xc8d56c6 thru 0xc8e0441 (0xad7c bytes) 2024-09-13T09:07:46.969 INFO:teuthology.orchestra.run.smithi031.stdout:1566 read 0xd5205d5 thru 0xd529098 (0x8ac4 bytes) 2024-09-13T09:07:46.991 INFO:teuthology.orchestra.run.smithi031.stdout:1567 trunc from 0xe7d66f8 to 0x51d4173 2024-09-13T09:07:47.011 INFO:teuthology.orchestra.run.smithi031.stdout:1568 read 0x4e99 thru 0xceab (0x8013 bytes) 2024-09-13T09:07:47.028 INFO:teuthology.orchestra.run.smithi031.stdout:1569 trunc from 0x51d4173 to 0xbbbfaa6 2024-09-13T09:07:47.038 INFO:teuthology.orchestra.run.smithi031.stdout:1570 clone 15 order 22 su 131072 sc 15 2024-09-13T09:07:47.042 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:47.040+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:47.090 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:47.090 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:47.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:47.157 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:47.154+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:47.232 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:47.232 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:47.244 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:47.240+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:47.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:47.344 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:47.342+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:47.408 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:47.408 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:47.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:47.464 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:47.462+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:47.546 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:47.542+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:47.586 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:47.587 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:47.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:47.646 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:47.642+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:47.747 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:47.742+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:47.772 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:47.772 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:47.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:47.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:07:47.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:47.878 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:47.876+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:47.932 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:47.932 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:47.949 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:47.944+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:47.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:48.033 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:48.033 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:48.049 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:48.046+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:48.052 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:48.052 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:48.063 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-13T09:07:48.072 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:48.072 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:48.099 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:48.100 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:48.148 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:48.148 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:48.148 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:48.150 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:48.148+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:48.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:48.178 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:48.178 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:48.199 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:48.199 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:48.217 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:48.217 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:48.235 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:48.236 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:48.244 INFO:teuthology.orchestra.run.smithi031.stdout:1571 read 0x3280042 thru 0x328b38b (0xb34a bytes) 2024-09-13T09:07:48.245 INFO:teuthology.orchestra.run.smithi031.stdout:1572 read 0x6076cbb thru 0x607d646 (0x698c bytes) 2024-09-13T09:07:48.246 INFO:teuthology.orchestra.run.smithi031.stdout:1574 write 0x1536f43 thru 0x153e1ab (0x7269 bytes) 2024-09-13T09:07:48.252 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:48.246+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:48.254 INFO:teuthology.orchestra.run.smithi031.stdout:1575 writesame 0x216310a thru 0x216b3b6 data_size 0x51(0x82ad bytes) 2024-09-13T09:07:48.260 INFO:teuthology.orchestra.run.smithi031.stdout:1576 write 0x6e2cacb thru 0x6e34481 (0x79b7 bytes) 2024-09-13T09:07:48.264 INFO:teuthology.orchestra.run.smithi031.stdout:1577 write 0x7cefda thru 0x7d72dd (0x8304 bytes) 2024-09-13T09:07:48.269 INFO:teuthology.orchestra.run.smithi031.stdout:1578 punch from 0x93e5165 to 0x93e9d15, (0x4bb0 bytes) 2024-09-13T09:07:48.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.275 INFO:teuthology.orchestra.run.smithi031.stdout:1580 write 0xbe4c67b thru 0xbe5a9e9 (0xe36f bytes) 2024-09-13T09:07:48.280 INFO:teuthology.orchestra.run.smithi031.stdout:1582 read 0x687cad0 thru 0x6886bb1 (0xa0e2 bytes) 2024-09-13T09:07:48.280 INFO:teuthology.orchestra.run.smithi031.stdout:1583 read 0x77258ff thru 0x77274c2 (0x1bc4 bytes) 2024-09-13T09:07:48.280 INFO:teuthology.orchestra.run.smithi031.stdout:1584 punch from 0x55aa779 to 0x55b975f, (0xefe6 bytes) 2024-09-13T09:07:48.281 INFO:teuthology.orchestra.run.smithi031.stdout:1585 writesame 0x28788ab thru 0x28859f3 data_size 0x3(0xd149 bytes) 2024-09-13T09:07:48.302 INFO:teuthology.orchestra.run.smithi031.stdout:1587 writesame 0xc962c52 thru 0xc96305c data_size 0x1(0x40b bytes) 2024-09-13T09:07:48.307 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:48.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:15159 3.c 3:33b101a6:::rbd_data.11242e26750a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15794176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.012154497000001, 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052673589999999999, 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 15159 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294458+0000", 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:48.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302503+0000", 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080450420000000005 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302534+0000", 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1352000000000001e-05 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302550+0000", 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5511000000000001e-05 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302617+0000", 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7722e-05 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302628+0000", 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0997e-05 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304649+0000", 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020203460000000001 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304734+0000", 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4839999999999994e-05 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T09:07:48.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.305520+0000", 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078672300000000002 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.343824+0000", 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038304108000000003 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347049+0000", 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032249399999999999 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347102+0000", 2024-09-13T09:07:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2163000000000002e-05 2024-09-13T09:07:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347133+0000", 2024-09-13T09:07:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1674999999999997e-05 2024-09-13T09:07:48.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20975 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4112384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.954407702999999, 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052586704999999997, 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20975 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352209+0000", 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.886e-06 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352216+0000", 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4159999999999998e-06 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352222+0000", 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404606+0000", 2024-09-13T09:07:48.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052384209000000001 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404695+0000", 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8455000000000006e-05 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404793+0000", 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8443999999999995e-05 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20976 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.954386938000001, 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052710422999999999, 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20976 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352230+0000", 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4829999999999999e-06 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352238+0000", 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.14e-06 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352245+0000", 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3130000000000004e-06 2024-09-13T09:07:48.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404836+0000", 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052590779999999997 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404881+0000", 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5034000000000003e-05 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404938+0000", 2024-09-13T09:07:48.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7120999999999997e-05 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20977 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4227072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.954364319, 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052908033, 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20977 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:48.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352249+0000", 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352259+0000", 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9520000000000006e-06 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352268+0000", 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5550000000000003e-06 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352274+0000", 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404981+0000", 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052706483999999998 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405058+0000", 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7282999999999999e-05 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405158+0000", 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9699999999999998e-05 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20978 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.954307756, 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053008803, 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20978 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352305+0000", 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352312+0000", 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5339999999999996e-06 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352325+0000", 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2952000000000001e-05 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352333+0000", 2024-09-13T09:07:48.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0180000000000001e-06 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405203+0000", 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052870013 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405257+0000", 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4135000000000003e-05 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405315+0000", 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8471999999999999e-05 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20979 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4308992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.954274713, 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053200718000000001, 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20979 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:48.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352342+0000", 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7610000000000001e-06 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352349+0000", 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352355+0000", 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405359+0000", 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053004355000000003 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405440+0000", 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0927999999999995e-05 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405540+0000", 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010015599999999999 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.337 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20980 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.954253781, 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053319995000000002, 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20980 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352362+0000", 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5959999999999998e-06 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352367+0000", 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4560000000000002e-06 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352370+0000", 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0860000000000002e-06 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405580+0000", 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053209583999999997 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405619+0000", 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9617999999999999e-05 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405680+0000", 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1259000000000004e-05 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.341 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20981 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4423680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.954241668, 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053531664999999999, 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20981 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.342 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352372+0000", 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:48.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352374+0000", 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5519999999999999e-06 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352377+0000", 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.258e-06 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352388+0000", 2024-09-13T09:07:48.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1375e-05 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405720+0000", 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053331770000000001 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405800+0000", 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0050999999999996e-05 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405904+0000", 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103879 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20982 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4456448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.954222894000001, 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053719282, 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20982 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352393+0000", 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.615e-06 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352396+0000", 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6090000000000001e-06 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352399+0000", 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405931+0000", 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053532140999999998 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406011+0000", 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0226000000000001e-05 2024-09-13T09:07:48.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406111+0000", 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9087999999999997e-05 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20983 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4505600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.953988516999999, 2024-09-13T09:07:48.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053724453999999998, 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20983 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352401+0000", 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997754 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352631+0000", 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000230134 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352644+0000", 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2308e-05 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352651+0000", 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2030000000000003e-06 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406162+0000", 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053510765000000002 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406241+0000", 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9764000000000004e-05 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406350+0000", 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010872199999999999 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20992 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4931584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950764196, 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052559458000000003, 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20992 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355849+0000", 2024-09-13T09:07:48.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355852+0000", 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2109999999999998e-06 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355859+0000", 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355864+0000", 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.028e-06 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408296+0000", 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052431658999999999 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408350+0000", 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4253999999999999e-05 2024-09-13T09:07:48.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408409+0000", 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9247999999999999e-05 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20993 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5013504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950724490000001, 2024-09-13T09:07:48.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052710119999999999, 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20993 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355889+0000", 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355893+0000", 2024-09-13T09:07:48.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2509999999999996e-06 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355903+0000", 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355909+0000", 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0190000000000003e-06 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408441+0000", 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052532395000000003 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408515+0000", 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4090000000000004e-05 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408600+0000", 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4716999999999995e-05 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20994 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5046272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950700114, 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052842215999999997, 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20994 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355913+0000", 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355916+0000", 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355923+0000", 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355931+0000", 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5280000000000003e-06 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408630+0000", 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052698997999999997 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408698+0000", 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7966999999999995e-05 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408756+0000", 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8233000000000003e-05 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20995 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5095424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950677067000001, 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052955222000000003, 2024-09-13T09:07:48.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20995 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355940+0000", 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1669999999999999e-06 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355946+0000", 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355951+0000", 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9309999999999999e-06 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408783+0000", 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052831751000000003 2024-09-13T09:07:48.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408838+0000", 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5031000000000002e-05 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408892+0000", 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4134e-05 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20996 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5128192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950658073, 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053075004000000002, 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20996 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:48.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355959+0000", 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355965+0000", 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355972+0000", 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408922+0000", 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052950174000000003 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408973+0000", 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0837000000000001e-05 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409031+0000", 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8455000000000002e-05 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20997 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5210112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950638025, 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053241038999999997, 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20997 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355979+0000", 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0170000000000001e-06 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355985+0000", 2024-09-13T09:07:48.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.122e-06 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355990+0000", 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409061+0000", 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053071209000000001 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409127+0000", 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5704999999999995e-05 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409217+0000", 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0492e-05 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20998 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5242880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950619617999999, 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053395182999999999, 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20998 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:48.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355994+0000", 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355997+0000", 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7470000000000001e-06 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356003+0000", 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356009+0000", 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1990000000000003e-06 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409244+0000", 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053234721999999998 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409321+0000", 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6998999999999999e-05 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409390+0000", 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8943999999999996e-05 2024-09-13T09:07:48.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20999 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5292032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950600562, 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053549106999999999, 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20999 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356013+0000", 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356016+0000", 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.016e-06 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356023+0000", 2024-09-13T09:07:48.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3810000000000004e-06 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356038+0000", 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5305999999999999e-05 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409408+0000", 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053369650999999997 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409472+0000", 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4587999999999999e-05 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409563+0000", 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0566e-05 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:21000 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950569588, 2024-09-13T09:07:48.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053705747999999998, 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21000 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356044+0000", 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356048+0000", 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.985e-06 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356056+0000", 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356061+0000", 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409590+0000", 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053528986000000001 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409663+0000", 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3342000000000004e-05 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409750+0000", 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7349999999999995e-05 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:21001 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950546234000001, 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053808105000000002, 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21001 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:48.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356067+0000", 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356080+0000", 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2228999999999999e-05 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356088+0000", 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356093+0000", 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:07:48.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409777+0000", 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053683869000000002 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409824+0000", 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6814000000000003e-05 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409876+0000", 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2405999999999999e-05 2024-09-13T09:07:48.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.399 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:48.399 INFO:teuthology.orchestra.run.smithi031.stdout:1588 punch from 0x3d81134 to 0x3d88872, (0x773e bytes) 2024-09-13T09:07:48.400 INFO:teuthology.orchestra.run.smithi031.stdout:1589 write 0x2efaf23 thru 0x2f095fb (0xe6d9 bytes) 2024-09-13T09:07:48.400 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.400 INFO:teuthology.orchestra.run.smithi031.stdout:1591 trunc from 0xc96305d to 0xa71858a 2024-09-13T09:07:48.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:48.401 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.401 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.401 INFO:teuthology.orchestra.run.smithi031.stdout:1594 write 0xa08b7b1 thru 0xa09121d (0x5a6d bytes) 2024-09-13T09:07:48.401 INFO:teuthology.orchestra.run.smithi031.stdout:1595 read 0xa4a3e0f thru 0xa4a6df5 (0x2fe7 bytes) 2024-09-13T09:07:48.402 INFO:teuthology.orchestra.run.smithi031.stdout:1596 trunc from 0xa71858a to 0xa38a7e3 2024-09-13T09:07:48.402 INFO:teuthology.orchestra.run.smithi031.stdout:1597 trunc from 0xa38a7e3 to 0x14c04b8 2024-09-13T09:07:48.447 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:48.444+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:48.454 INFO:teuthology.orchestra.run.smithi031.stdout:1598 trunc from 0x14c04b8 to 0xd4132c0 2024-09-13T09:07:48.464 INFO:teuthology.orchestra.run.smithi031.stdout:1600 read 0x1906c8d thru 0x190c219 (0x558d bytes) 2024-09-13T09:07:48.465 INFO:teuthology.orchestra.run.smithi031.stdout:1601 read 0x7c727f7 thru 0x7c7f85f (0xd069 bytes) 2024-09-13T09:07:48.465 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.465 INFO:teuthology.orchestra.run.smithi031.stdout:1603 write 0x72f961b thru 0x72fe597 (0x4f7d bytes) 2024-09-13T09:07:48.495 INFO:teuthology.orchestra.run.smithi031.stdout:1604 trunc from 0xd4132c0 to 0x19af975 2024-09-13T09:07:48.504 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:48.502+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:48.535 INFO:teuthology.orchestra.run.smithi031.stdout:1608 write 0x570bb79 thru 0x5711f06 (0x638e bytes) 2024-09-13T09:07:48.543 INFO:teuthology.orchestra.run.smithi031.stdout:1609 read 0x51e53d1 thru 0x51e8c2b (0x385b bytes) 2024-09-13T09:07:48.544 INFO:teuthology.orchestra.run.smithi031.stdout:1611 read 0x4d729e thru 0x4dd0b5 (0x5e18 bytes) 2024-09-13T09:07:48.545 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.545 INFO:teuthology.orchestra.run.smithi031.stdout:1614 read 0x33008fe thru 0x3304a54 (0x4157 bytes) 2024-09-13T09:07:48.548 INFO:teuthology.orchestra.run.smithi031.stdout:1615 trunc from 0x5711f07 to 0x6fa2f2a 2024-09-13T09:07:48.558 INFO:teuthology.orchestra.run.smithi031.stdout:1617 punch from 0x57a0343 to 0x57a64b9, (0x6176 bytes) 2024-09-13T09:07:48.558 INFO:teuthology.orchestra.run.smithi031.stdout:1618 writesame 0x52c4fc3 thru 0x52cb918 data_size 0x2(0x6956 bytes) 2024-09-13T09:07:48.589 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:48.589 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:48.589 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:48.600 INFO:teuthology.orchestra.run.smithi031.stdout:1619 write 0xe4ada94 thru 0xe4b1810 (0x3d7d bytes) 2024-09-13T09:07:48.605 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:48.602+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:48.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:48.607 INFO:teuthology.orchestra.run.smithi031.stdout:1620 writesame 0x743d7ac thru 0x7445433 data_size 0x14(0x7c88 bytes) 2024-09-13T09:07:48.615 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.615 INFO:teuthology.orchestra.run.smithi031.stdout:1623 read 0x9b17a99 thru 0x9b24b01 (0xd069 bytes) 2024-09-13T09:07:48.615 INFO:teuthology.orchestra.run.smithi031.stdout:1625 write 0xa2b3f78 thru 0xa2c3a96 (0xfb1f bytes) 2024-09-13T09:07:48.623 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.623 INFO:teuthology.orchestra.run.smithi031.stdout:1629 writesame 0xba0efb0 thru 0xba10126 data_size 0x1(0x1177 bytes) 2024-09-13T09:07:48.631 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.631 INFO:teuthology.orchestra.run.smithi031.stdout:1631 punch from 0x62840f4 to 0x628a3ec, (0x62f8 bytes) 2024-09-13T09:07:48.631 INFO:teuthology.orchestra.run.smithi031.stdout:1632 write 0x34c445f thru 0x34c6d83 (0x2925 bytes) 2024-09-13T09:07:48.635 INFO:teuthology.orchestra.run.smithi031.stdout:1633 read 0x82e1dbf thru 0x82e3511 (0x1753 bytes) 2024-09-13T09:07:48.636 INFO:teuthology.orchestra.run.smithi031.stdout:1637 writesame 0x4c16497 thru 0x4c1f8bb data_size 0x29(0x9425 bytes) 2024-09-13T09:07:48.642 INFO:teuthology.orchestra.run.smithi031.stdout:1638 writesame 0x8568bbb thru 0x85691ee data_size 0x2(0x634 bytes) 2024-09-13T09:07:48.648 INFO:teuthology.orchestra.run.smithi031.stdout:1639 punch from 0x28bd0ac to 0x28beaa4, (0x19f8 bytes) 2024-09-13T09:07:48.648 INFO:teuthology.orchestra.run.smithi031.stdout:1640 writesame 0x9c448dd thru 0x9c45afd data_size 0x33(0x1221 bytes) 2024-09-13T09:07:48.655 INFO:teuthology.orchestra.run.smithi031.stdout:1642 write 0x4f5d6fd thru 0x4f6b9de (0xe2e2 bytes) 2024-09-13T09:07:48.661 INFO:teuthology.orchestra.run.smithi031.stdout:1643 writesame 0xac4456f thru 0xac518d8 data_size 0x1(0xd36a bytes) 2024-09-13T09:07:48.670 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:48.670 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:07:48.671 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-13T09:07:48.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-13T09:07:48.700 INFO:teuthology.orchestra.run.smithi031.stdout:1645 trunc from 0xe4b1811 to 0xc1a1d55 2024-09-13T09:07:48.706 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:48.704+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:48.736 INFO:teuthology.orchestra.run.smithi031.stdout:1646 trunc from 0xc1a1d55 to 0xcbad07 2024-09-13T09:07:48.744 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:48.745 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:48.745 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:48.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:48.774 INFO:teuthology.orchestra.run.smithi031.stdout:1647 trunc from 0xcbad07 to 0x4c1095b 2024-09-13T09:07:48.782 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.782 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.782 INFO:teuthology.orchestra.run.smithi031.stdout:1650 read 0x2bbb496 thru 0x2bc2431 (0x6f9c bytes) 2024-09-13T09:07:48.783 INFO:teuthology.orchestra.run.smithi031.stdout:1651 punch from 0x1775374 to 0x177d08e, (0x7d1a bytes) 2024-09-13T09:07:48.783 INFO:teuthology.orchestra.run.smithi031.stdout:1653 punch from 0x348ac36 to 0x348dfdc, (0x33a6 bytes) 2024-09-13T09:07:48.783 INFO:teuthology.orchestra.run.smithi031.stdout:1655 read 0x4235c5f thru 0x4242249 (0xc5eb bytes) 2024-09-13T09:07:48.783 INFO:teuthology.orchestra.run.smithi031.stdout:1656 punch from 0x1acc4be to 0x1acc999, (0x4db bytes) 2024-09-13T09:07:48.784 INFO:teuthology.orchestra.run.smithi031.stdout:1657 writesame 0x24ba058 thru 0x24bc2ff data_size 0x4(0x22a8 bytes) 2024-09-13T09:07:48.787 INFO:teuthology.orchestra.run.smithi031.stdout:1659 punch from 0x405b65c to 0x4066200, (0xaba4 bytes) 2024-09-13T09:07:48.787 INFO:teuthology.orchestra.run.smithi031.stdout:1660 write 0x1a6c08c thru 0x1a73fc6 (0x7f3b bytes) 2024-09-13T09:07:48.789 INFO:teuthology.orchestra.run.smithi031.stdout:1661 read 0x2100565 thru 0x2102ccd (0x2769 bytes) 2024-09-13T09:07:48.807 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:48.804+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:48.812 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:07:48.815 INFO:teuthology.orchestra.run.smithi031.stdout:1662 write 0xdc4582d thru 0xdc53fde (0xe7b2 bytes) 2024-09-13T09:07:48.824 INFO:teuthology.orchestra.run.smithi031.stdout:1664 writesame 0x44b92aa thru 0x44bbea5 data_size 0xa(0x2bfc bytes) 2024-09-13T09:07:48.831 INFO:teuthology.orchestra.run.smithi031.stdout:1665 read 0xa84d7bf thru 0xa858210 (0xaa52 bytes) 2024-09-13T09:07:48.831 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.831 INFO:teuthology.orchestra.run.smithi031.stdout:1668 write 0x2d41c13 thru 0x2d4c874 (0xac62 bytes) 2024-09-13T09:07:48.838 INFO:teuthology.orchestra.run.smithi031.stdout:1669 punch from 0x2594bc1 to 0x25a1b85, (0xcfc4 bytes) 2024-09-13T09:07:48.838 INFO:teuthology.orchestra.run.smithi031.stdout:1670 writesame 0x2c6f490 thru 0x2c737a5 data_size 0x2(0x4316 bytes) 2024-09-13T09:07:48.843 INFO:teuthology.orchestra.run.smithi031.stdout:1671 write 0x3dad779 thru 0x3db1569 (0x3df1 bytes) 2024-09-13T09:07:48.849 INFO:teuthology.orchestra.run.smithi031.stdout:1674 punch from 0xd0ed0a6 to 0xd0f3e6a, (0x6dc4 bytes) 2024-09-13T09:07:48.849 INFO:teuthology.orchestra.run.smithi031.stdout:1675 write 0x46b725b thru 0x46b82dd (0x1083 bytes) 2024-09-13T09:07:48.855 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.855 INFO:teuthology.orchestra.run.smithi031.stdout:1677 write 0x2ce284f thru 0x2ce8674 (0x5e26 bytes) 2024-09-13T09:07:48.860 INFO:teuthology.orchestra.run.smithi031.stdout:1679 read 0x71c6cc thru 0x71d064 (0x999 bytes) 2024-09-13T09:07:48.860 INFO:teuthology.orchestra.run.smithi031.stdout:1680 write 0x40a2a16 thru 0x40b286b (0xfe56 bytes) 2024-09-13T09:07:48.866 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:48.872 INFO:teuthology.orchestra.run.smithi031.stdout:1682 trunc from 0xdc53fdf to 0x985f26a 2024-09-13T09:07:48.891 INFO:teuthology.orchestra.run.smithi031.stdout:1683 writesame 0x2310f0 thru 0x236ecd data_size 0x1b(0x5dde bytes) 2024-09-13T09:07:48.895 INFO:teuthology.orchestra.run.smithi031.stdout:1684 write 0x89f497 thru 0x8ab268 (0xbdd2 bytes) 2024-09-13T09:07:48.900 INFO:teuthology.orchestra.run.smithi031.stdout:1686 trunc from 0x985f26a to 0xaa2a5ec 2024-09-13T09:07:48.908 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:48.906+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:48.909 INFO:teuthology.orchestra.run.smithi031.stdout:1687 read 0x45c3be6 thru 0x45c434c (0x767 bytes) 2024-09-13T09:07:48.923 INFO:teuthology.orchestra.run.smithi031.stdout:1688 trunc from 0xaa2a5ec to 0x6a4b21 2024-09-13T09:07:48.936 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.239688518999998, 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840076, 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 34 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694022+0000", 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694033+0000", 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.116e-05 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694051+0000", 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694065+0000", 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3716e-05 2024-09-13T09:07:48.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700273+0000", 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062082320000000002 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700405+0000", 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131416 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700608+0000", 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020368099999999999 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705556+0000", 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049475760000000004 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705654+0000", 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8444999999999997e-05 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705741+0000", 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6416999999999996e-05 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705863+0000", 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121876 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.239620682000002, 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:48.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:48.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.239312667999997, 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.239224755000002, 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:48.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:48.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:48.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:48.960 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.968700394000003, 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:48.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:48.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.968612630999999, 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.968560578000002, 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:48.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.968375496, 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.968301007000001, 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:48.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:48.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.968246686999997, 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:48.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:48.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:48.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.968048602000003, 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:48.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:48.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:49.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.002 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.967967338999998, 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:49.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:49.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:49.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.473672624000002, 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:49.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:49.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:49.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.473356987000003, 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.473296656999999, 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:49.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:49.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.473238750999997, 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:49.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:49.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:49.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.472936154000003, 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:49.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:49.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:49.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:49.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7649 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.080370986999998, 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013081786, 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7649 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853339+0000", 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853345+0000", 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853365+0000", 2024-09-13T09:07:49.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9054000000000001e-05 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853374+0000", 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858418+0000", 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050442869999999997 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858525+0000", 2024-09-13T09:07:49.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010602099999999999 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858992+0000", 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046758299999999999 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.861712+0000", 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027195700000000001 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866315+0000", 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046037439999999999 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866381+0000", 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5121000000000002e-05 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866422+0000", 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1331000000000001e-05 2024-09-13T09:07:49.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7657 3.10 3:08eae27d:::rbd_data.11242ae5e73a.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13762560~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.065578692999999, 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018927840000000001, 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7657 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868131+0000", 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870832+0000", 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027013660000000002 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870853+0000", 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.107e-05 2024-09-13T09:07:49.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870862+0000", 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7830000000000004e-06 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870910+0000", 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871083+0000", 2024-09-13T09:07:49.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017277500000000001 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871386+0000", 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030378600000000001 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878951+0000", 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075642649999999997 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886982+0000", 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080316160000000001 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887025+0000", 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3288999999999998e-05 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887060+0000", 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4706999999999998e-05 2024-09-13T09:07:49.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7660 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.062835604, 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016257230000000001, 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7660 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870874+0000", 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870879+0000", 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870885+0000", 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870888+0000", 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876842+0000", 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059541780000000001 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876952+0000", 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109577 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.877351+0000", 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039971899999999999 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878995+0000", 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001643179 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887073+0000", 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080780109999999995 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887100+0000", 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7585000000000001e-05 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887133+0000", 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2533000000000002e-05 2024-09-13T09:07:49.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.057 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:49.058 INFO:teuthology.orchestra.run.smithi031.stdout:1689 read 0x6946e0 thru 0x69c066 (0x7987 bytes) 2024-09-13T09:07:49.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:49.060 INFO:teuthology.orchestra.run.smithi031.stdout:1690 write 0x93f6ebf thru 0x94026d6 (0xb818 bytes) 2024-09-13T09:07:49.060 INFO:teuthology.orchestra.run.smithi031.stdout:1691 write 0xc9a33a6 thru 0xc9aef56 (0xbbb1 bytes) 2024-09-13T09:07:49.060 INFO:teuthology.orchestra.run.smithi031.stdout:1692 read 0xa41be38 thru 0xa420c94 (0x4e5d bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1694 write 0x5e14fe5 thru 0x5e1a03e (0x505a bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1695 write 0x8cd8738 thru 0x8ce587c (0xd145 bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1696 read 0x75a4687 thru 0x75a63b8 (0x1d32 bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1699 punch from 0x38b6e47 to 0x38c641d, (0xf5d6 bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1700 punch from 0x23623d0 to 0x236cc80, (0xa8b0 bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1701 writesame 0x4c7f4af thru 0x4c867d6 data_size 0x6e(0x7328 bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1704 write 0xe9875fb thru 0xe98ce4e (0x5854 bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1706 read 0x9672e9b thru 0x967da59 (0xabbf bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1708 read 0xc03c16f thru 0xc04ab8e (0xea20 bytes) 2024-09-13T09:07:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:1711 write 0xaebeb9 thru 0xafb543 (0xf68b bytes) 2024-09-13T09:07:49.062 INFO:teuthology.orchestra.run.smithi031.stdout:1714 clone 16 order 23 su 131072 sc 3 2024-09-13T09:07:49.062 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-13T09:07:49.044+0000 7f426cbbd640 -1 mon.a@0(leader).osd e48 definitely_dead 0 2024-09-13T09:07:49.062 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:49.054+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:49.160 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:49.154+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:49.228 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:49.228 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:49.228 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:49.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:49.260 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:49.258+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:49.361 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:49.356+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:49.372 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:49.372 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:49.372 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:49.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:49.462 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:49.458+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:49.561 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220334748999999, 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:07:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220317976, 2024-09-13T09:07:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:07:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220280670999999, 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:07:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220260957000001, 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:07:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:07:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220244632, 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:07:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:07:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220228766, 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:07:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220212981, 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:07:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:07:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220193910000001, 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:07:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:07:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220177362999999, 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:07:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:07:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220149764, 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220132522, 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:07:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220102079, 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:07:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:07:49.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220083164, 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:07:49.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220065376000001, 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220040151999999, 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:07:49.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:07:49.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220021947999999, 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:07:49.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.220001630000001, 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:07:49.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:07:49.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.219983082000001, 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:07:49.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:07:49.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.219965808, 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.219948354, 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:49.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:49.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:07:49.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:49.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:49.652 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:49.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:49.654 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:49.648+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:49.755 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:49.752+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:49.822 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.1. 2024-09-13T09:07:49.837 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:49.837 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:49.837 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:49.851 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-13T09:07:49.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:49.895 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:49.892+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:49.956 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:49.954+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:50.020 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:50.021 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:50.021 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:50.021 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:50.021 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:50.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:50.081 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:50.076+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:50.158 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:50.154+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.024588991999998, 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:07:50.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:07:50.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:07:50.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14831 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.918780979000001, 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071499264000000007, 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14831 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236541+0000", 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3910000000000001e-06 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236548+0000", 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236553+0000", 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307720+0000", 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071166574999999996 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307883+0000", 2024-09-13T09:07:50.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000163809 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308039+0000", 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155229 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14834 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 606208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917458312000001, 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070919408000000003, 2024-09-13T09:07:50.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14834 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237862+0000", 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237864+0000", 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.694e-06 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237871+0000", 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237877+0000", 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2989999999999996e-06 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308614+0000", 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070737169000000003 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308704+0000", 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0357000000000002e-05 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308782+0000", 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7480999999999999e-05 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14835 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:50.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917423087, 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071108525000000006, 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14835 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237908+0000", 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1191e-05 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237929+0000", 2024-09-13T09:07:50.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1189e-05 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237936+0000", 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308827+0000", 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070891320999999993 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308924+0000", 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6665999999999999e-05 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309006+0000", 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2231000000000006e-05 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14836 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917378110000001, 2024-09-13T09:07:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071290567999999999, 2024-09-13T09:07:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14836 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:50.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237945+0000", 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6229999999999999e-06 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237953+0000", 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237958+0000", 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0309999999999998e-06 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309041+0000", 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071083576999999995 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309132+0000", 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0500000000000004e-05 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309233+0000", 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101116 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14837 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917358304, 2024-09-13T09:07:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071483336999999994, 2024-09-13T09:07:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14837 2024-09-13T09:07:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:50.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237964+0000", 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237971+0000", 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237975+0000", 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6360000000000002e-06 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309280+0000", 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071304765000000006 2024-09-13T09:07:50.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309359+0000", 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9066999999999994e-05 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309445+0000", 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6255999999999995e-05 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14838 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917338714, 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071691981000000002, 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14838 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:50.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237981+0000", 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237984+0000", 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237990+0000", 2024-09-13T09:07:50.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237995+0000", 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309487+0000", 2024-09-13T09:07:50.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071492752000000007 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309575+0000", 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7106000000000003e-05 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309674+0000", 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9208000000000003e-05 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14844 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917321281, 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072094264000000005, 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14844 2024-09-13T09:07:50.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:50.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238002+0000", 2024-09-13T09:07:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T09:07:50.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238008+0000", 2024-09-13T09:07:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.032e-06 2024-09-13T09:07:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238013+0000", 2024-09-13T09:07:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:07:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309739+0000", 2024-09-13T09:07:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071726390000000001 2024-09-13T09:07:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309786+0000", 2024-09-13T09:07:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6561999999999999e-05 2024-09-13T09:07:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310093+0000", 2024-09-13T09:07:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030766000000000002 2024-09-13T09:07:50.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14845 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.91730312, 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072306747000000005, 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14845 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:50.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238019+0000", 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238025+0000", 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4870000000000002e-06 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238029+0000", 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3769999999999998e-06 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310139+0000", 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072110056000000006 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310224+0000", 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4291999999999999e-05 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310324+0000", 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100405 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14846 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1343488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917285356000001, 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072637322000000004, 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14846 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238037+0000", 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238045+0000", 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0960000000000001e-06 2024-09-13T09:07:50.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238051+0000", 2024-09-13T09:07:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T09:07:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310365+0000", 2024-09-13T09:07:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072313575000000005 2024-09-13T09:07:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310432+0000", 2024-09-13T09:07:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6888000000000001e-05 2024-09-13T09:07:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310672+0000", 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024050699999999999 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14847 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917264225, 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072846102999999995, 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14847 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238058+0000", 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5119999999999998e-06 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238063+0000", 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238067+0000", 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4750000000000004e-06 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310717+0000", 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072649243000000002 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310803+0000", 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6147e-05 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310902+0000", 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9474000000000003e-05 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14848 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:50.220 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.917248594, 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072985396999999994, 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14848 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.221 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238074+0000", 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.227e-06 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238080+0000", 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238085+0000", 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1410000000000004e-06 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310939+0000", 2024-09-13T09:07:50.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072853862000000005 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310997+0000", 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8248000000000002e-05 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311057+0000", 2024-09-13T09:07:50.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0071999999999997e-05 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14849 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.916314099000001, 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072287373000000002, 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14849 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238089+0000", 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9990826 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239012+0000", 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092334199999999996 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239026+0000", 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3351e-05 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239034+0000", 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4419999999999998e-06 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311104+0000", 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072069464 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311194+0000", 2024-09-13T09:07:50.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9922999999999999e-05 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311294+0000", 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001002 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14850 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3702784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:50.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.916278137999999, 2024-09-13T09:07:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072393710999999999, 2024-09-13T09:07:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14850 2024-09-13T09:07:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239045+0000", 2024-09-13T09:07:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4429999999999999e-06 2024-09-13T09:07:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239053+0000", 2024-09-13T09:07:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5179999999999998e-06 2024-09-13T09:07:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239058+0000", 2024-09-13T09:07:50.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311330+0000", 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072271613999999998 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311381+0000", 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0908000000000001e-05 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311436+0000", 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5513000000000002e-05 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.239 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14861 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.916055191000002, 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072582646000000001, 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14861 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239270+0000", 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239280+0000", 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0210999999999999e-05 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239285+0000", 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.967e-06 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311481+0000", 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072196249000000004 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311573+0000", 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1588999999999993e-05 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311848+0000", 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027526699999999999 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14862 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.916031880999999, 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072869149999999994, 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14862 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239288+0000", 2024-09-13T09:07:50.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239295+0000", 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239302+0000", 2024-09-13T09:07:50.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.793e-06 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239305+0000", 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9380000000000001e-06 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311878+0000", 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072573031999999996 2024-09-13T09:07:50.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311923+0000", 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4351000000000003e-05 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312158+0000", 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023512399999999999 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14863 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.916012617, 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072990372999999997, 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14863 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239309+0000", 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939999999999999e-06 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239312+0000", 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239315+0000", 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7630000000000001e-06 2024-09-13T09:07:50.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312186+0000", 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072871378000000001 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312243+0000", 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6487000000000003e-05 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312298+0000", 2024-09-13T09:07:50.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5506e-05 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14864 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.915952807, 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073255022000000003, 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14864 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239317+0000", 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999949 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239372+0000", 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5457000000000001e-05 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239381+0000", 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.003e-06 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239387+0000", 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6130000000000003e-06 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312327+0000", 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072940489999999997 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312409+0000", 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.153e-05 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312623+0000", 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213799 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14865 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.915682282999999, 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073307136999999994, 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.258 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14865 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:50.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239390+0000", 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999752 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239643+0000", 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025296200000000002 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239653+0000", 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7839999999999998e-06 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239660+0000", 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0570000000000004e-06 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312655+0000", 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072994857999999996 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312731+0000", 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5754999999999996e-05 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312945+0000", 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021457799999999999 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14866 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.915654112999999, 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073614097000000003, 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14866 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:50.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:50.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239669+0000", 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.858e-06 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239675+0000", 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:50.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239680+0000", 2024-09-13T09:07:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:07:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312975+0000", 2024-09-13T09:07:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073294697000000006 2024-09-13T09:07:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313051+0000", 2024-09-13T09:07:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6297999999999994e-05 2024-09-13T09:07:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:50.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313280+0000", 2024-09-13T09:07:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022920800000000001 2024-09-13T09:07:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:50.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:50.269 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:50.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:50.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:50.271 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:50.266+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:50.372 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:50.368+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:50.405 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:50.405 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:50.405 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:50.405 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:50.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:50.470 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:50.471 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:50.472 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:50.470+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:50.487 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:50.487 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:50.503 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:50.503 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:50.520 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:50.520 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:50.540 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:50.540 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:50.549 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:50.549 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:50.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:50.561 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:50.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:50.573 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:50.568+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:50.581 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:50.582 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:50.602 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:50.602 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:50.674 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:50.670+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:50.732 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:50.732 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:50.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:50.788 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:50.786+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:50.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:07:50.876 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:50.870+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:50.927 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:50.927 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:50.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:50.988 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:50.986+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:51.077 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:51.074+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:51.127 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:51.127 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:51.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:51.185 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:51.181+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:51.279 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:51.273+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:51.313 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:51.313 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:51.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:51.379 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:51.376+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:51.480 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:51.477+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:51.500 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:51.500 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:51.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:51.581 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:51.576+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:51.682 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:51.680+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:51.688 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:51.688 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:51.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:51.783 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:51.780+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:51.828 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:07:51.832 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:51.832 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:51.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:51.892 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:51.889+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:51.965 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:51.965 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:51.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:52.024 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:52.021+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:52.086 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:52.081+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:52.150 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:52.150 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:52.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:52.187 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:52.184+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:52.287 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:52.284+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:52.327 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:52.328 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:52.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:52.388 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:52.385+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:52.476 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:52.476 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:52.489 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:52.485+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:52.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:52.590 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:52.584+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:52.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:52.626 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:52.626 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:52.642 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone15 2024-09-13T09:07:52.647 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:52.691 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:52.685+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:52.792 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:52.789+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:52.804 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:52.804 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:52.832 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:52.832 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:52.838 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:15159 3.c 3:33b101a6:::rbd_data.11242e26750a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 15794176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.543541607999998, 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052673589999999999, 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 15159 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294460+0000", 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.294458+0000", 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302503+0000", 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080450420000000005 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302534+0000", 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1352000000000001e-05 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302550+0000", 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5511000000000001e-05 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302617+0000", 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7722e-05 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-09-13T09:07:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.302628+0000", 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0997e-05 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304649+0000", 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020203460000000001 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.304734+0000", 2024-09-13T09:07:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4839999999999994e-05 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.305520+0000", 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078672300000000002 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.343824+0000", 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.038304108000000003 2024-09-13T09:07:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347049+0000", 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032249399999999999 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347102+0000", 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2163000000000002e-05 2024-09-13T09:07:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.347133+0000", 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1674999999999997e-05 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20975 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4112384~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485794813999998, 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052586704999999997, 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20975 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352206+0000", 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352209+0000", 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.886e-06 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352216+0000", 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4159999999999998e-06 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352222+0000", 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6849999999999998e-06 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404606+0000", 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052384209000000001 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404695+0000", 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8455000000000006e-05 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404793+0000", 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8443999999999995e-05 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20976 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4145152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485774049, 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052710422999999999, 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20976 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352227+0000", 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352230+0000", 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4829999999999999e-06 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352238+0000", 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.14e-06 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352245+0000", 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3130000000000004e-06 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404836+0000", 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052590779999999997 2024-09-13T09:07:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404881+0000", 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5034000000000003e-05 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404938+0000", 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7120999999999997e-05 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20977 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4227072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485751430000001, 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052908033, 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20977 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352250+0000", 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352249+0000", 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352259+0000", 2024-09-13T09:07:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9520000000000006e-06 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352268+0000", 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5550000000000003e-06 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352274+0000", 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4439999999999996e-06 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.404981+0000", 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052706483999999998 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405058+0000", 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7282999999999999e-05 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405158+0000", 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9699999999999998e-05 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20978 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485694866999999, 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053008803, 2024-09-13T09:07:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20978 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352306+0000", 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352305+0000", 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352312+0000", 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5339999999999996e-06 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352325+0000", 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2952000000000001e-05 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352333+0000", 2024-09-13T09:07:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0180000000000001e-06 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405203+0000", 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052870013 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405257+0000", 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4135000000000003e-05 2024-09-13T09:07:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405315+0000", 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8471999999999999e-05 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20979 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4308992~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485661824000001, 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053200718000000001, 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20979 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352339+0000", 2024-09-13T09:07:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352342+0000", 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7610000000000001e-06 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352349+0000", 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0639999999999996e-06 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352355+0000", 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3789999999999997e-06 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405359+0000", 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053004355000000003 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405440+0000", 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0927999999999995e-05 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405540+0000", 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010015599999999999 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20980 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485640891999999, 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053319995000000002, 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20980 2024-09-13T09:07:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352360+0000", 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352362+0000", 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5959999999999998e-06 2024-09-13T09:07:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352367+0000", 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4560000000000002e-06 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352370+0000", 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0860000000000002e-06 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405580+0000", 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053209583999999997 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405619+0000", 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9617999999999999e-05 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405680+0000", 2024-09-13T09:07:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1259000000000004e-05 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20981 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4423680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485628778999999, 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053531664999999999, 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20981 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352373+0000", 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352372+0000", 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352374+0000", 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5519999999999999e-06 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352377+0000", 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.258e-06 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352388+0000", 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1375e-05 2024-09-13T09:07:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405720+0000", 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053331770000000001 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405800+0000", 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0050999999999996e-05 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405904+0000", 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103879 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20982 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4456448~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485610005000002, 2024-09-13T09:07:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053719282, 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20982 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352391+0000", 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352393+0000", 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.615e-06 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352396+0000", 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6090000000000001e-06 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352399+0000", 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.405931+0000", 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053532140999999998 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406011+0000", 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0226000000000001e-05 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406111+0000", 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9087999999999997e-05 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20983 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4505600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.485375628, 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053724453999999998, 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20983 2024-09-13T09:07:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352626+0000", 2024-09-13T09:07:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352401+0000", 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997754 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352631+0000", 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000230134 2024-09-13T09:07:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352644+0000", 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2308e-05 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.352651+0000", 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2030000000000003e-06 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406162+0000", 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053510765000000002 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406241+0000", 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9764000000000004e-05 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.406350+0000", 2024-09-13T09:07:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010872199999999999 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20992 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 4931584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.482151306999999, 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052559458000000003, 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20992 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355850+0000", 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355849+0000", 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355852+0000", 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2109999999999998e-06 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355859+0000", 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355864+0000", 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.028e-06 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408296+0000", 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052431658999999999 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408350+0000", 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4253999999999999e-05 2024-09-13T09:07:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408409+0000", 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9247999999999999e-05 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20993 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5013504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.482111601, 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052710119999999999, 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20993 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355890+0000", 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355889+0000", 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355893+0000", 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2509999999999996e-06 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355903+0000", 2024-09-13T09:07:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2830000000000007e-06 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355909+0000", 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0190000000000003e-06 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408441+0000", 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052532395000000003 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408515+0000", 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4090000000000004e-05 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408600+0000", 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4716999999999995e-05 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20994 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5046272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.482087225000001, 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052842215999999997, 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20994 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355914+0000", 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355913+0000", 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355916+0000", 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0960000000000001e-06 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355923+0000", 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355931+0000", 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5280000000000003e-06 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408630+0000", 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052698997999999997 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408698+0000", 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7966999999999995e-05 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408756+0000", 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8233000000000003e-05 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20995 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5095424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.482064178000002, 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052955222000000003, 2024-09-13T09:07:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20995 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355937+0000", 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355940+0000", 2024-09-13T09:07:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1669999999999999e-06 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355946+0000", 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6370000000000002e-06 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355951+0000", 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9309999999999999e-06 2024-09-13T09:07:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408783+0000", 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052831751000000003 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408838+0000", 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5031000000000002e-05 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408892+0000", 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4134e-05 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20996 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5128192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.482045184, 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053075004000000002, 2024-09-13T09:07:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20996 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355956+0000", 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355959+0000", 2024-09-13T09:07:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8650000000000001e-06 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355965+0000", 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5899999999999996e-06 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355972+0000", 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408922+0000", 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.052950174000000003 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.408973+0000", 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0837000000000001e-05 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409031+0000", 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8455000000000002e-05 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20997 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5210112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.482025136000001, 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053241038999999997, 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20997 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355976+0000", 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355979+0000", 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0170000000000001e-06 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355985+0000", 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.122e-06 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355990+0000", 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409061+0000", 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053071209000000001 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409127+0000", 2024-09-13T09:07:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5704999999999995e-05 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409217+0000", 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0492e-05 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20998 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5242880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.482006728999998, 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053395182999999999, 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20998 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355995+0000", 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355994+0000", 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.355997+0000", 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7470000000000001e-06 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356003+0000", 2024-09-13T09:07:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356009+0000", 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1990000000000003e-06 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409244+0000", 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053234721999999998 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409321+0000", 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6998999999999999e-05 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409390+0000", 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8943999999999996e-05 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20999 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5292032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.481987672999999, 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053549106999999999, 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20999 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356014+0000", 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356013+0000", 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356016+0000", 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.016e-06 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356023+0000", 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3810000000000004e-06 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356038+0000", 2024-09-13T09:07:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5305999999999999e-05 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409408+0000", 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053369650999999997 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409472+0000", 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4587999999999999e-05 2024-09-13T09:07:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409563+0000", 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0566e-05 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:21000 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.481956699000001, 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053705747999999998, 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21000 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356045+0000", 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356044+0000", 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356048+0000", 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.985e-06 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356056+0000", 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1389999999999995e-06 2024-09-13T09:07:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356061+0000", 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409590+0000", 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053528986000000001 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409663+0000", 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3342000000000004e-05 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409750+0000", 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7349999999999995e-05 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:21001 3.1c 3:3a5b2b5e:::rbd_data.1124d0fab35.0000000000000007:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.481933345000002, 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053808105000000002, 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21001 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356068+0000", 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356067+0000", 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356080+0000", 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2228999999999999e-05 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356088+0000", 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.356093+0000", 2024-09-13T09:07:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409777+0000", 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.053683869000000002 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409824+0000", 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6814000000000003e-05 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.409876+0000", 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2405999999999999e-05 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:52.949 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:52.953 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:52.954 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:52.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:52.955 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-13T09:07:52.955 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:52.956 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:52.956 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:52.957 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:52.957 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:52.957 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:52.958 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:52.948+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:52.960 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:53.054 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:53.048+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3.5e900490 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.11915424600000001, 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.11919640099999999, 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:07:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:07:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:07:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-13T09:07:53.075 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:53.084 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:53.158 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:53.153+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:53.253 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:53.254 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:53.254 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:53.258 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:53.253+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:53.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:53.363 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:53.360+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:53.398 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:53.398 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:53.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:53.398 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.700923052, 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840076, 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 34 2024-09-13T09:07:53.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694023+0000", 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694022+0000", 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694033+0000", 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.116e-05 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694051+0000", 2024-09-13T09:07:53.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8559999999999998e-05 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694065+0000", 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3716e-05 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700273+0000", 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062082320000000002 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700405+0000", 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131416 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.700608+0000", 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020368099999999999 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705556+0000", 2024-09-13T09:07:53.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049475760000000004 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705654+0000", 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8444999999999997e-05 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705741+0000", 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6416999999999996e-05 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705863+0000", 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121876 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.700855214999997, 2024-09-13T09:07:53.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:53.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:53.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.700547200999999, 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:53.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:53.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:53.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.700459287999998, 2024-09-13T09:07:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:53.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.429934926999998, 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:53.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:53.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.429847164000002, 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:53.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:53.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:53.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.429795110999997, 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.438 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:53.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.429610029000003, 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:53.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:53.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:53.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.429535540000003, 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:53.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:53.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:53.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.42948122, 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.456 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:53.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.429283134999999, 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:53.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:53.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:53.468 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.429201872, 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:53.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:53.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:53.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.934907156999998, 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:53.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:53.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.934591519999998, 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:53.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:53.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.934531190000001, 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.934473283999999, 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.934170686999998, 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7649 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.541605520000001, 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013081786, 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7649 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853339+0000", 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853345+0000", 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:07:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853365+0000", 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9054000000000001e-05 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853374+0000", 2024-09-13T09:07:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858418+0000", 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050442869999999997 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858525+0000", 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010602099999999999 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858992+0000", 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046758299999999999 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.861712+0000", 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027195700000000001 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866315+0000", 2024-09-13T09:07:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046037439999999999 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866381+0000", 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5121000000000002e-05 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866422+0000", 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1331000000000001e-05 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7657 3.10 3:08eae27d:::rbd_data.11242ae5e73a.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13762560~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.526813226000002, 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018927840000000001, 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7657 2024-09-13T09:07:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868131+0000", 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870832+0000", 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027013660000000002 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870853+0000", 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.107e-05 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870862+0000", 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7830000000000004e-06 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870910+0000", 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871083+0000", 2024-09-13T09:07:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017277500000000001 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871386+0000", 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030378600000000001 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878951+0000", 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075642649999999997 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886982+0000", 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080316160000000001 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887025+0000", 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3288999999999998e-05 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887060+0000", 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4706999999999998e-05 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7660 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.524070136999999, 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016257230000000001, 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7660 2024-09-13T09:07:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870874+0000", 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870879+0000", 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870885+0000", 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:07:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870888+0000", 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876842+0000", 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059541780000000001 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876952+0000", 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109577 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.877351+0000", 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039971899999999999 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878995+0000", 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001643179 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887073+0000", 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080780109999999995 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887100+0000", 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7585000000000001e-05 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887133+0000", 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2533000000000002e-05 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.524 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:53.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:53.526 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:53.521+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:53.626 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:53.621+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:53.696 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:53.696 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:53.696 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:53.696 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:53.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:53.727 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:53.721+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:53.828 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:53.821+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:53.833 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:53.833 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:53.833 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:53.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:53.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:07:53.928 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:53.925+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:53.973 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:53.973 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:53.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:53.973 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:53.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:53.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:53.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631251249999998, 2024-09-13T09:07:53.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:07:53.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:53.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:53.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631234477, 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:07:53.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:07:53.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:07:53.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:07:53.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:07:53.985 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631197172, 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:07:53.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631177458, 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:53.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:07:53.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:07:53.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:07:53.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:53.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:53.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631161132999999, 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:53.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:53.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:54.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:07:54.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:07:54.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:07:54.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:07:54.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:07:54.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:07:54.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:07:54.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:07:54.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631145267000001, 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:07:54.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631129481999999, 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:07:54.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:54.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:54.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:54.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:07:54.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:07:54.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:07:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:07:54.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:07:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:07:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631110411000002, 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:07:54.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:54.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:07:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631093864, 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:07:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:07:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:07:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:07:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:07:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:07:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631066265000001, 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:07:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631049022999999, 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:07:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:07:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:07:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:07:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:07:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.631018579999999, 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:07:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:07:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.630999665000001, 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:07:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:07:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:07:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.630981877, 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:07:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:07:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:07:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:07:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.630956652999998, 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.630938448999999, 2024-09-13T09:07:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:07:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.630918131000001, 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:07:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.630899583000001, 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:07:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:07:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.630882309, 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:07:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.630864854999999, 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:07:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:07:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.094 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:54.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:54.097 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:54.089+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:54.123 INFO:teuthology.orchestra.run.smithi031.stdout:1716 trunc from 0xe98ce4f to 0x7085aeb 2024-09-13T09:07:54.164 INFO:teuthology.orchestra.run.smithi031.stdout:1718 writesame 0xbca1ee6 thru 0xbcae2c9 data_size 0x3f(0xc3e4 bytes) 2024-09-13T09:07:54.164 INFO:teuthology.orchestra.run.smithi031.stdout:1719 write 0xad097c5 thru 0xad0c207 (0x2a43 bytes) 2024-09-13T09:07:54.169 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.169 INFO:teuthology.orchestra.run.smithi031.stdout:1722 write 0xb380352 thru 0xb380dba (0xa69 bytes) 2024-09-13T09:07:54.176 INFO:teuthology.orchestra.run.smithi031.stdout:1723 read 0x65fcb5d thru 0x6600c46 (0x40ea bytes) 2024-09-13T09:07:54.177 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.177 INFO:teuthology.orchestra.run.smithi031.stdout:1726 punch from 0x6aa0a9 to 0x6ac259, (0x21b0 bytes) 2024-09-13T09:07:54.177 INFO:teuthology.orchestra.run.smithi031.stdout:1727 punch from 0x9ea58d4 to 0x9eb09e6, (0xb112 bytes) 2024-09-13T09:07:54.177 INFO:teuthology.orchestra.run.smithi031.stdout:1729 write 0x92d086f thru 0x92dc2f8 (0xba8a bytes) 2024-09-13T09:07:54.182 INFO:teuthology.orchestra.run.smithi031.stdout:1730 writesame 0xba17018 thru 0xba1f12e data_size 0x1(0x8117 bytes) 2024-09-13T09:07:54.197 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:54.192+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:54.209 INFO:teuthology.orchestra.run.smithi031.stdout:1731 write 0x2604132 thru 0x260c9bd (0x888c bytes) 2024-09-13T09:07:54.218 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.218 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.218 INFO:teuthology.orchestra.run.smithi031.stdout:1738 write 0x194cb5 thru 0x196946 (0x1c92 bytes) 2024-09-13T09:07:54.230 INFO:teuthology.orchestra.run.smithi031.stdout:1740 read 0x7734ea7 thru 0x7742c40 (0xdd9a bytes) 2024-09-13T09:07:54.231 INFO:teuthology.orchestra.run.smithi031.stdout:1741 write 0x42cc241 thru 0x42d4284 (0x8044 bytes) 2024-09-13T09:07:54.234 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:54.234 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:54.234 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:54.238 INFO:teuthology.orchestra.run.smithi031.stdout:1742 write 0x4d035e2 thru 0x4d08f14 (0x5933 bytes) 2024-09-13T09:07:54.244 INFO:teuthology.orchestra.run.smithi031.stdout:1743 read 0x2e32bc9 thru 0x2e356e4 (0x2b1c bytes) 2024-09-13T09:07:54.244 INFO:teuthology.orchestra.run.smithi031.stdout:1745 read 0x1a6789d thru 0x1a718ec (0xa050 bytes) 2024-09-13T09:07:54.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:54.262 INFO:teuthology.orchestra.run.smithi031.stdout:1746 writesame 0xdd5fcef thru 0xdd6944c data_size 0x7d(0x975e bytes) 2024-09-13T09:07:54.262 INFO:teuthology.orchestra.run.smithi031.stdout:1748 read 0xdb50d81 thru 0xdb57a46 (0x6cc6 bytes) 2024-09-13T09:07:54.262 INFO:teuthology.orchestra.run.smithi031.stdout:1749 write 0x225d0dc thru 0x2265bf3 (0x8b18 bytes) 2024-09-13T09:07:54.269 INFO:teuthology.orchestra.run.smithi031.stdout:1750 write 0xc072b33 thru 0xc07465f (0x1b2d bytes) 2024-09-13T09:07:54.273 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.273 INFO:teuthology.orchestra.run.smithi031.stdout:1752 writesame 0xb514aad thru 0xb519688 data_size 0xa(0x4bdc bytes) 2024-09-13T09:07:54.277 INFO:teuthology.orchestra.run.smithi031.stdout:1754 punch from 0x71cf6b0 to 0x71d6bae, (0x74fe bytes) 2024-09-13T09:07:54.277 INFO:teuthology.orchestra.run.smithi031.stdout:1755 read 0x818fa7f thru 0x8191624 (0x1ba6 bytes) 2024-09-13T09:07:54.277 INFO:teuthology.orchestra.run.smithi031.stdout:1756 read 0xc6c3537 thru 0xc6c77fd (0x42c7 bytes) 2024-09-13T09:07:54.278 INFO:teuthology.orchestra.run.smithi031.stdout:1757 write 0xc6872bb thru 0xc68916d (0x1eb3 bytes) 2024-09-13T09:07:54.282 INFO:teuthology.orchestra.run.smithi031.stdout:1759 write 0x1e177d2 thru 0x1e1c05f (0x488e bytes) 2024-09-13T09:07:54.288 INFO:teuthology.orchestra.run.smithi031.stdout:1760 writesame 0x72da121 thru 0x72dfe5e data_size 0x6e(0x5d3e bytes) 2024-09-13T09:07:54.291 INFO:teuthology.orchestra.run.smithi031.stdout:1761 read 0x6d05c7c thru 0x6d0e741 (0x8ac6 bytes) 2024-09-13T09:07:54.292 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.296 INFO:teuthology.orchestra.run.smithi031.stdout:1765 trunc from 0xdd6944d to 0xa8b0d62 2024-09-13T09:07:54.297 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:54.293+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:54.317 INFO:teuthology.orchestra.run.smithi031.stdout:1767 trunc from 0xa8b0d62 to 0x9fc5fbf 2024-09-13T09:07:54.337 INFO:teuthology.orchestra.run.smithi031.stdout:1768 read 0x279097d thru 0x2797529 (0x6bad bytes) 2024-09-13T09:07:54.338 INFO:teuthology.orchestra.run.smithi031.stdout:1769 write 0x12f9eb2 thru 0x12fdceb (0x3e3a bytes) 2024-09-13T09:07:54.343 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.355 INFO:teuthology.orchestra.run.smithi031.stdout:1772 writesame 0xb936d6f thru 0xb93a266 data_size 0x1e(0x34f8 bytes) 2024-09-13T09:07:54.356 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.356 INFO:teuthology.orchestra.run.smithi031.stdout:1774 write 0xa95cf3f thru 0xa967002 (0xa0c4 bytes) 2024-09-13T09:07:54.361 INFO:teuthology.orchestra.run.smithi031.stdout:1775 read 0x519a41f thru 0x519aa57 (0x639 bytes) 2024-09-13T09:07:54.362 INFO:teuthology.orchestra.run.smithi031.stdout:1776 write 0x2af153b thru 0x2afb2e3 (0x9da9 bytes) 2024-09-13T09:07:54.364 INFO:teuthology.orchestra.run.smithi031.stdout:1777 writesame 0x76fcde0 thru 0x76fe908 data_size 0x1(0x1b29 bytes) 2024-09-13T09:07:54.375 INFO:teuthology.orchestra.run.smithi031.stdout:1780 trunc from 0xb93a267 to 0xddb036a 2024-09-13T09:07:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:1781 read 0x7f23bcf thru 0x7f24edc (0x130e bytes) 2024-09-13T09:07:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:1782 read 0xc0065f8 thru 0xc00ed66 (0x876f bytes) 2024-09-13T09:07:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:1783 read 0x7b45339 thru 0x7b535a0 (0xe268 bytes) 2024-09-13T09:07:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:1784 writesame 0x2f6b5ab thru 0x2f71352 data_size 0x6f(0x5da8 bytes) 2024-09-13T09:07:54.389 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:54.390 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:54.390 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:54.390 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:54.390 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:54.390 INFO:teuthology.orchestra.run.smithi031.stdout:1785 read 0xbaa181b thru 0xbaa728b (0x5a71 bytes) 2024-09-13T09:07:54.392 INFO:teuthology.orchestra.run.smithi031.stdout:1788 punch from 0x3eb8c63 to 0x3ebd6a9, (0x4a46 bytes) 2024-09-13T09:07:54.392 INFO:teuthology.orchestra.run.smithi031.stdout:1790 writesame 0x60a276d thru 0x60b0bd7 data_size 0x5(0xe46b bytes) 2024-09-13T09:07:54.398 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:54.393+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:54.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:54.419 INFO:teuthology.orchestra.run.smithi031.stdout:1791 writesame 0xec5c81c thru 0xec648c9 data_size 0x1a(0x80ae bytes) 2024-09-13T09:07:54.419 INFO:teuthology.orchestra.run.smithi031.stdout:1792 read 0xa8948c thru 0xa906be (0x7233 bytes) 2024-09-13T09:07:54.420 INFO:teuthology.orchestra.run.smithi031.stdout:1793 write 0x323fd69 thru 0x32413c9 (0x1661 bytes) 2024-09-13T09:07:54.426 INFO:teuthology.orchestra.run.smithi031.stdout:1794 write 0x9b183c0 thru 0x9b20d0a (0x894b bytes) 2024-09-13T09:07:54.431 INFO:teuthology.orchestra.run.smithi031.stdout:1795 write 0x4421d2c thru 0x44283af (0x6684 bytes) 2024-09-13T09:07:54.435 INFO:teuthology.orchestra.run.smithi031.stdout:1796 write 0xb7d67ba thru 0xb7e3633 (0xce7a bytes) 2024-09-13T09:07:54.439 INFO:teuthology.orchestra.run.smithi031.stdout:1797 write 0xfc0e00 thru 0xfc4ecb (0x40cc bytes) 2024-09-13T09:07:54.442 INFO:teuthology.orchestra.run.smithi031.stdout:1798 read 0x7721c45 thru 0x772718d (0x5549 bytes) 2024-09-13T09:07:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:1799 read 0x80ae6f7 thru 0x80b62ab (0x7bb5 bytes) 2024-09-13T09:07:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:1800 read 0xd712d0b thru 0xd71cb12 (0x9e08 bytes) 2024-09-13T09:07:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:1802 read 0x563bbdc thru 0x563d13e (0x1563 bytes) 2024-09-13T09:07:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:1803 read 0x585d29c thru 0x5867e4e (0xabb3 bytes) 2024-09-13T09:07:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:1805 writesame 0xcc16ac5 thru 0xcc1b53b data_size 0x1(0x4a77 bytes) 2024-09-13T09:07:54.470 INFO:teuthology.orchestra.run.smithi031.stdout:1806 read 0x3bd2797 thru 0x3bd85ca (0x5e34 bytes) 2024-09-13T09:07:54.471 INFO:teuthology.orchestra.run.smithi031.stdout:1808 writesame 0x830cc4e thru 0x8319a7d data_size 0x8(0xce30 bytes) 2024-09-13T09:07:54.485 INFO:teuthology.orchestra.run.smithi031.stdout:1809 punch from 0x9302e81 to 0x93124b9, (0xf638 bytes) 2024-09-13T09:07:54.485 INFO:teuthology.orchestra.run.smithi031.stdout:1810 write 0xd7785ce thru 0xd77f252 (0x6c85 bytes) 2024-09-13T09:07:54.491 INFO:teuthology.orchestra.run.smithi031.stdout:1812 write 0x78463e0 thru 0x7846bea (0x80b bytes) 2024-09-13T09:07:54.497 INFO:teuthology.orchestra.run.smithi031.stdout:1813 punch from 0xd7946da to 0xd7a1c78, (0xd59e bytes) 2024-09-13T09:07:54.497 INFO:teuthology.orchestra.run.smithi031.stdout:1815 write 0x6b4018e thru 0x6b46cf1 (0x6b64 bytes) 2024-09-13T09:07:54.499 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:54.493+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:54.500 INFO:teuthology.orchestra.run.smithi031.stdout:1816 read 0x34dacb6 thru 0x34e6b44 (0xbe8f bytes) 2024-09-13T09:07:54.502 INFO:teuthology.orchestra.run.smithi031.stdout:1817 punch from 0xe58214 to 0xe6023d, (0x8029 bytes) 2024-09-13T09:07:54.502 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.502 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.502 INFO:teuthology.orchestra.run.smithi031.stdout:1821 write 0x5faae46 thru 0x5fae744 (0x38ff bytes) 2024-09-13T09:07:54.510 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.510 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.511 INFO:teuthology.orchestra.run.smithi031.stdout:1826 writesame 0xaa3782f thru 0xaa44fc3 data_size 0x1(0xd795 bytes) 2024-09-13T09:07:54.542 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:54.542 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:54.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:54.542 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:54.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:07:54.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:54.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.409092475999998, 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:07:54.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:54.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14831 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.303284463000001, 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071499264000000007, 2024-09-13T09:07:54.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14831 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236541+0000", 2024-09-13T09:07:54.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3910000000000001e-06 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236548+0000", 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236553+0000", 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307720+0000", 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071166574999999996 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307883+0000", 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000163809 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308039+0000", 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155229 2024-09-13T09:07:54.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14835 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.301926570999999, 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071108525000000006, 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14835 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237908+0000", 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1191e-05 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237929+0000", 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1189e-05 2024-09-13T09:07:54.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237936+0000", 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308827+0000", 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070891320999999993 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308924+0000", 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6665999999999999e-05 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309006+0000", 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2231000000000006e-05 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14836 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:54.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.301881594000001, 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071290567999999999, 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14836 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237945+0000", 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6229999999999999e-06 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237953+0000", 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237958+0000", 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0309999999999998e-06 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309041+0000", 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071083576999999995 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309132+0000", 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0500000000000004e-05 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309233+0000", 2024-09-13T09:07:54.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101116 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14837 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.301861788, 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071483336999999994, 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14837 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237964+0000", 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237971+0000", 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237975+0000", 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6360000000000002e-06 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309280+0000", 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071304765000000006 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309359+0000", 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9066999999999994e-05 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309445+0000", 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6255999999999995e-05 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14838 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.301842197999999, 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071691981000000002, 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14838 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237981+0000", 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237984+0000", 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237990+0000", 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237995+0000", 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309487+0000", 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071492752000000007 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309575+0000", 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7106000000000003e-05 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309674+0000", 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9208000000000003e-05 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14844 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.301824764999999, 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072094264000000005, 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14844 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:54.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238002+0000", 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238008+0000", 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.032e-06 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238013+0000", 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309739+0000", 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071726390000000001 2024-09-13T09:07:54.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309786+0000", 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6561999999999999e-05 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310093+0000", 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030766000000000002 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14845 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.301806603999999, 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072306747000000005, 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14845 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238019+0000", 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238025+0000", 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4870000000000002e-06 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238029+0000", 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3769999999999998e-06 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310139+0000", 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072110056000000006 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310224+0000", 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4291999999999999e-05 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310324+0000", 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100405 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14846 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1343488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.30178884, 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072637322000000004, 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14846 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:54.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238037+0000", 2024-09-13T09:07:54.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238045+0000", 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0960000000000001e-06 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238051+0000", 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310365+0000", 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072313575000000005 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310432+0000", 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6888000000000001e-05 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310672+0000", 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024050699999999999 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14847 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.301767709, 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072846102999999995, 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14847 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:54.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238058+0000", 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5119999999999998e-06 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238063+0000", 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238067+0000", 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4750000000000004e-06 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310717+0000", 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072649243000000002 2024-09-13T09:07:54.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310803+0000", 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6147e-05 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310902+0000", 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9474000000000003e-05 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14848 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.301752078, 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072985396999999994, 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14848 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238074+0000", 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.227e-06 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238080+0000", 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238085+0000", 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1410000000000004e-06 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310939+0000", 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072853862000000005 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310997+0000", 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8248000000000002e-05 2024-09-13T09:07:54.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311057+0000", 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0071999999999997e-05 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14849 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.300817583000001, 2024-09-13T09:07:54.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072287373000000002, 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14849 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238089+0000", 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9990826 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239012+0000", 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092334199999999996 2024-09-13T09:07:54.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239026+0000", 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3351e-05 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239034+0000", 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4419999999999998e-06 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311104+0000", 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072069464 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311194+0000", 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9922999999999999e-05 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311294+0000", 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001002 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14850 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3702784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.300781621999999, 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072393710999999999, 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14850 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.602 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239045+0000", 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4429999999999999e-06 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239053+0000", 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5179999999999998e-06 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239058+0000", 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311330+0000", 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072271613999999998 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311381+0000", 2024-09-13T09:07:54.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0908000000000001e-05 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311436+0000", 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5513000000000002e-05 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14861 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.300558675000001, 2024-09-13T09:07:54.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072582646000000001, 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14861 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239270+0000", 2024-09-13T09:07:54.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239280+0000", 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0210999999999999e-05 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239285+0000", 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.967e-06 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311481+0000", 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072196249000000004 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311573+0000", 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1588999999999993e-05 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311848+0000", 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027526699999999999 2024-09-13T09:07:54.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14862 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.300535365000002, 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072869149999999994, 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14862 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239288+0000", 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239295+0000", 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239302+0000", 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.793e-06 2024-09-13T09:07:54.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239305+0000", 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9380000000000001e-06 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311878+0000", 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072573031999999996 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311923+0000", 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4351000000000003e-05 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312158+0000", 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023512399999999999 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14863 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.300516100999999, 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072990372999999997, 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14863 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239309+0000", 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939999999999999e-06 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239312+0000", 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239315+0000", 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7630000000000001e-06 2024-09-13T09:07:54.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312186+0000", 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072871378000000001 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312243+0000", 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6487000000000003e-05 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312298+0000", 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5506e-05 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14864 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.300456291, 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073255022000000003, 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14864 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:54.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239317+0000", 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999949 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239372+0000", 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5457000000000001e-05 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239381+0000", 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.003e-06 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239387+0000", 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6130000000000003e-06 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312327+0000", 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072940489999999997 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312409+0000", 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.153e-05 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312623+0000", 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213799 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14865 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.300185766999999, 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073307136999999994, 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14865 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239390+0000", 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999752 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239643+0000", 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025296200000000002 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239653+0000", 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7839999999999998e-06 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239660+0000", 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0570000000000004e-06 2024-09-13T09:07:54.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312655+0000", 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072994857999999996 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312731+0000", 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5754999999999996e-05 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312945+0000", 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021457799999999999 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14866 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:54.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.300157596999998, 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073614097000000003, 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14866 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239669+0000", 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.858e-06 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239675+0000", 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239680+0000", 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312975+0000", 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073294697000000006 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313051+0000", 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6297999999999994e-05 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313280+0000", 2024-09-13T09:07:54.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022920800000000001 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5808102960000001, 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:54.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:54.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:54.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:54.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:54.639 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:54.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:54.641 INFO:teuthology.orchestra.run.smithi031.stdout:1827 read 0x7a9fec4 thru 0x7aa636e (0x64ab bytes) 2024-09-13T09:07:54.642 INFO:teuthology.orchestra.run.smithi031.stdout:1828 read 0x88f8c9a thru 0x89049bd (0xbd24 bytes) 2024-09-13T09:07:54.642 INFO:teuthology.orchestra.run.smithi031.stdout:1829 writesame 0xcc8ea62 thru 0xcc9259c data_size 0x1(0x3b3b bytes) 2024-09-13T09:07:54.642 INFO:teuthology.orchestra.run.smithi031.stdout:1831 write 0xcf79097 thru 0xcf8752d (0xe497 bytes) 2024-09-13T09:07:54.642 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.642 INFO:teuthology.orchestra.run.smithi031.stdout:1833 read 0x5972203 thru 0x59786bf (0x64bd bytes) 2024-09-13T09:07:54.642 INFO:teuthology.orchestra.run.smithi031.stdout:1834 trunc from 0xec648ca to 0x605d3fa 2024-09-13T09:07:54.642 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:54.637+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:54.647 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.647 INFO:teuthology.orchestra.run.smithi031.stdout:1837 read 0x43419c5 thru 0x434d460 (0xba9c bytes) 2024-09-13T09:07:54.670 INFO:teuthology.orchestra.run.smithi031.stdout:1838 write 0xa1e4e26 thru 0xa1e877e (0x3959 bytes) 2024-09-13T09:07:54.677 INFO:teuthology.orchestra.run.smithi031.stdout:1839 write 0x132eadd thru 0x13341c4 (0x56e8 bytes) 2024-09-13T09:07:54.681 INFO:teuthology.orchestra.run.smithi031.stdout:1842 read 0x8a1170f thru 0x8a210c6 (0xf9b8 bytes) 2024-09-13T09:07:54.681 INFO:teuthology.orchestra.run.smithi031.stdout:1843 read 0x87b067b thru 0x87b75dc (0x6f62 bytes) 2024-09-13T09:07:54.697 INFO:teuthology.orchestra.run.smithi031.stdout:1845 writesame 0xb109cce thru 0xb10a0d0 data_size 0x1(0x403 bytes) 2024-09-13T09:07:54.697 INFO:teuthology.orchestra.run.smithi031.stdout:1846 write 0xa3b7d04 thru 0xa3b876b (0xa68 bytes) 2024-09-13T09:07:54.703 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.703 INFO:teuthology.orchestra.run.smithi031.stdout:1848 read 0x8113f53 thru 0x811c4df (0x858d bytes) 2024-09-13T09:07:54.704 INFO:teuthology.orchestra.run.smithi031.stdout:1849 read 0x2eecf65 thru 0x2ef2f55 (0x5ff1 bytes) 2024-09-13T09:07:54.705 INFO:teuthology.orchestra.run.smithi031.stdout:1850 write 0xafac581 thru 0xafb6351 (0x9dd1 bytes) 2024-09-13T09:07:54.712 INFO:teuthology.orchestra.run.smithi031.stdout:1851 read 0x1f674f3 thru 0x1f6f7bd (0x82cb bytes) 2024-09-13T09:07:54.713 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.713 INFO:teuthology.orchestra.run.smithi031.stdout:1853 write 0x7a630de thru 0x7a6f3f5 (0xc318 bytes) 2024-09-13T09:07:54.720 INFO:teuthology.orchestra.run.smithi031.stdout:1854 write 0xbe6e3f thru 0xbf4821 (0xd9e3 bytes) 2024-09-13T09:07:54.730 INFO:teuthology.orchestra.run.smithi031.stdout:1855 write 0x2f6a828 thru 0x2f70bbf (0x6398 bytes) 2024-09-13T09:07:54.734 INFO:teuthology.orchestra.run.smithi031.stdout:1857 punch from 0xacf93d0 to 0xacfb719, (0x2349 bytes) 2024-09-13T09:07:54.734 INFO:teuthology.orchestra.run.smithi031.stdout:1858 write 0x98f92d6 thru 0x9908a51 (0xf77c bytes) 2024-09-13T09:07:54.741 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.741 INFO:teuthology.orchestra.run.smithi031.stdout:1860 read 0xac00ff6 thru 0xac05c04 (0x4c0f bytes) 2024-09-13T09:07:54.741 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.741 INFO:teuthology.orchestra.run.smithi031.stdout:1866 punch from 0x4a2158f to 0x4a2c3f8, (0xae69 bytes) 2024-09-13T09:07:54.741 INFO:teuthology.orchestra.run.smithi031.stdout:1867 write 0x781341d thru 0x7813af3 (0x6d7 bytes) 2024-09-13T09:07:54.742 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:54.737+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:54.765 INFO:teuthology.orchestra.run.smithi031.stdout:1868 write 0xc445388 thru 0xc449e13 (0x4a8c bytes) 2024-09-13T09:07:54.770 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.771 INFO:teuthology.orchestra.run.smithi031.stdout:1870 punch from 0x3b5f932 to 0x3b6f255, (0xf923 bytes) 2024-09-13T09:07:54.771 INFO:teuthology.orchestra.run.smithi031.stdout:1871 read 0x127c89a thru 0x127ee4b (0x25b2 bytes) 2024-09-13T09:07:54.772 INFO:teuthology.orchestra.run.smithi031.stdout:1873 punch from 0x28f2a5a to 0x28f987c, (0x6e22 bytes) 2024-09-13T09:07:54.772 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.781 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:54.781 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:54.781 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:54.781 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:54.790 INFO:teuthology.orchestra.run.smithi031.stdout:1876 trunc from 0xc449e14 to 0x5819432 2024-09-13T09:07:54.795 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:54.826 INFO:teuthology.orchestra.run.smithi031.stdout:1877 write 0x3ee5fce thru 0x3ee9e20 (0x3e53 bytes) 2024-09-13T09:07:54.842 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:54.840+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:54.846 INFO:teuthology.orchestra.run.smithi031.stdout:1879 trunc from 0x5819432 to 0xb84b97e 2024-09-13T09:07:54.852 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:07:54.852 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:07:54.852 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-13T09:07:54.852 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:07:54.852 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:07:54.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-13T09:07:54.858 INFO:teuthology.orchestra.run.smithi031.stdout:1881 read 0x68448f5 thru 0x68487d5 (0x3ee1 bytes) 2024-09-13T09:07:54.858 INFO:teuthology.orchestra.run.smithi031.stdout:1883 write 0xa0fe5a6 thru 0xa102978 (0x43d3 bytes) 2024-09-13T09:07:54.881 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:07:54.887 INFO:teuthology.orchestra.run.smithi031.stdout:1884 trunc from 0xb84b97e to 0x3efb4bf 2024-09-13T09:07:54.914 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:54.938 INFO:teuthology.orchestra.run.smithi031.stdout:1887 write 0xb8b8fec thru 0xb8bbb51 (0x2b66 bytes) 2024-09-13T09:07:54.943 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:54.937+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:54.955 INFO:teuthology.orchestra.run.smithi031.stdout:1888 write 0xcc84542 thru 0xcc8ba10 (0x74cf bytes) 2024-09-13T09:07:54.960 INFO:teuthology.orchestra.run.smithi031.stdout:1889 read 0x7e171f thru 0x7eb7d0 (0xa0b2 bytes) 2024-09-13T09:07:54.961 INFO:teuthology.orchestra.run.smithi031.stdout:1890 punch from 0x9fc32aa to 0x9fc725d, (0x3fb3 bytes) 2024-09-13T09:07:54.961 INFO:teuthology.orchestra.run.smithi031.stdout:1891 read 0x3a2090b thru 0x3a29dda (0x94d0 bytes) 2024-09-13T09:07:54.962 INFO:teuthology.orchestra.run.smithi031.stdout:1892 write 0xb0f8a17 thru 0xb1089fc (0xffe6 bytes) 2024-09-13T09:07:54.970 INFO:teuthology.orchestra.run.smithi031.stdout:1893 writesame 0xc800381 thru 0xc80a743 data_size 0x35(0xa3c3 bytes) 2024-09-13T09:07:54.973 INFO:teuthology.orchestra.run.smithi031.stdout:1894 write 0x8c1af9f thru 0x8c1e923 (0x3985 bytes) 2024-09-13T09:07:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:54.974 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:54.974 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:07:54.974 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:07:54.974 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:54.978 INFO:teuthology.orchestra.run.smithi031.stdout:1895 punch from 0xac9afed to 0xaca9fdb, (0xefee bytes) 2024-09-13T09:07:54.978 INFO:teuthology.orchestra.run.smithi031.stdout:1896 write 0x561e86e thru 0x5627a96 (0x9229 bytes) 2024-09-13T09:07:54.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:54.985 INFO:teuthology.orchestra.run.smithi031.stdout:1897 write 0x6e5768b thru 0x6e6467b (0xcff1 bytes) 2024-09-13T09:07:54.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:54.993 INFO:teuthology.orchestra.run.smithi031.stdout:1900 write 0x3f7772a thru 0x3f7fa64 (0x833b bytes) 2024-09-13T09:07:54.997 INFO:teuthology.orchestra.run.smithi031.stdout:1901 punch from 0x8f606ba to 0x8f609ea, (0x330 bytes) 2024-09-13T09:07:54.997 INFO:teuthology.orchestra.run.smithi031.stdout:1902 read 0x4dc88cb thru 0x4dd1a59 (0x918f bytes) 2024-09-13T09:07:54.998 INFO:teuthology.orchestra.run.smithi031.stdout:1903 punch from 0x15b5ac6 to 0x15b936a, (0x38a4 bytes) 2024-09-13T09:07:54.998 INFO:teuthology.orchestra.run.smithi031.stdout:1904 write 0xc826b4c thru 0xc82d33e (0x67f3 bytes) 2024-09-13T09:07:55.004 INFO:teuthology.orchestra.run.smithi031.stdout:1905 punch from 0x71b864d to 0x71c3016, (0xa9c9 bytes) 2024-09-13T09:07:55.004 INFO:teuthology.orchestra.run.smithi031.stdout:1906 punch from 0xbddb4a1 to 0xbde3a46, (0x85a5 bytes) 2024-09-13T09:07:55.004 INFO:teuthology.orchestra.run.smithi031.stdout:1908 write 0x465c2c0 thru 0x4667337 (0xb078 bytes) 2024-09-13T09:07:55.006 INFO:teuthology.orchestra.run.smithi031.stdout:1910 write 0x6f9d0c8 thru 0x6fa3561 (0x649a bytes) 2024-09-13T09:07:55.012 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:55.012 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:07:55.012 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:55.012 INFO:teuthology.orchestra.run.smithi031.stdout:1912 writesame 0x466f6ff thru 0x4671aa0 data_size 0x1(0x23a2 bytes) 2024-09-13T09:07:55.043 INFO:teuthology.orchestra.run.smithi031.stdout:1913 trunc from 0xcc8ba11 to 0x3201d53 2024-09-13T09:07:55.062 INFO:teuthology.orchestra.run.smithi031.stdout:1914 read 0x683ae3 thru 0x68712a (0x3648 bytes) 2024-09-13T09:07:55.063 INFO:teuthology.orchestra.run.smithi031.stdout:1915 read 0x4324bf thru 0x43783c (0x537e bytes) 2024-09-13T09:07:55.063 INFO:teuthology.orchestra.run.smithi031.stdout:1917 read 0x28f73bb thru 0x290119c (0x9de2 bytes) 2024-09-13T09:07:55.064 INFO:teuthology.orchestra.run.smithi031.stdout:1920 writesame 0x308ad74 thru 0x3090c24 data_size 0x1(0x5eb1 bytes) 2024-09-13T09:07:55.072 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:55.069+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:55.094 INFO:teuthology.orchestra.run.smithi031.stdout:1922 punch from 0x3900ee to 0x3918a7, (0x17b9 bytes) 2024-09-13T09:07:55.095 INFO:teuthology.orchestra.run.smithi031.stdout:1923 read 0x23a6504 thru 0x23b05b4 (0xa0b1 bytes) 2024-09-13T09:07:55.095 INFO:teuthology.orchestra.run.smithi031.stdout:1925 write 0x1ef6f7e thru 0x1ef8239 (0x12bc bytes) 2024-09-13T09:07:55.099 INFO:teuthology.orchestra.run.smithi031.stdout:1926 write 0x2788418 thru 0x278e213 (0x5dfc bytes) 2024-09-13T09:07:55.119 INFO:teuthology.orchestra.run.smithi031.stdout:1927 writesame 0x6db33e1 thru 0x6db7b6b data_size 0xf(0x478b bytes) 2024-09-13T09:07:55.120 INFO:teuthology.orchestra.run.smithi031.stdout:1928 read 0x58ce2c8 thru 0x58d3bfe (0x5937 bytes) 2024-09-13T09:07:55.120 INFO:teuthology.orchestra.run.smithi031.stdout:1929 read 0xfc4728 thru 0xfc9f0f (0x57e8 bytes) 2024-09-13T09:07:55.132 INFO:teuthology.orchestra.run.smithi031.stdout:1930 writesame 0x8daf5de thru 0x8dbc465 data_size 0xc(0xce88 bytes) 2024-09-13T09:07:55.133 INFO:teuthology.orchestra.run.smithi031.stdout:1931 read 0x544db1b thru 0x544f6e6 (0x1bcc bytes) 2024-09-13T09:07:55.143 INFO:teuthology.orchestra.run.smithi031.stdout:1932 write 0x98e7087 thru 0x98ecc71 (0x5beb bytes) 2024-09-13T09:07:55.144 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:55.140+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:55.144 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:55.144 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:55.146 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:55.146 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:55.146 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:55.146 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:07:55.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.146 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22488 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 6864896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695014+0000", 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.447130004999998, 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083619160999999997, 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22488 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695014+0000", 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695014+0000", 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695013+0000", 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695017+0000", 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4359999999999998e-06 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695025+0000", 2024-09-13T09:07:55.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4940000000000003e-06 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695032+0000", 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9890000000000003e-06 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778364+0000", 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083332099000000007 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778502+0000", 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013811300000000001 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778633+0000", 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130646 2024-09-13T09:07:55.152 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22489 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695039+0000", 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.447105317999998, 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083795781999999999, 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.153 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22489 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695039+0000", 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695039+0000", 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695038+0000", 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695050+0000", 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1769e-05 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695059+0000", 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1640000000000007e-06 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695066+0000", 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778681+0000", 2024-09-13T09:07:55.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083614843999999994 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778742+0000", 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1271000000000003e-05 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778834+0000", 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2270999999999998e-05 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22490 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695073+0000", 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.447071376, 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084084148999999997, 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22490 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695073+0000", 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695073+0000", 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695072+0000", 2024-09-13T09:07:55.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695075+0000", 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4300000000000002e-06 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695084+0000", 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6570000000000003e-06 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695091+0000", 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8170000000000001e-06 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778888+0000", 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.08379702 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779022+0000", 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013453799999999999 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779157+0000", 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439699999999999 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22491 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695097+0000", 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.447047196, 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084251518999999997, 2024-09-13T09:07:55.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22491 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695097+0000", 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695097+0000", 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695096+0000", 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695100+0000", 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4769999999999999e-06 2024-09-13T09:07:55.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695108+0000", 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5029999999999997e-06 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695115+0000", 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9800000000000001e-06 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779204+0000", 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084088642000000005 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779264+0000", 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0210999999999997e-05 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779348+0000", 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4308999999999995e-05 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22492 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7061504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695126+0000", 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.447017762000002, 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084554658000000005, 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.167 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22492 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695126+0000", 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695126+0000", 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695125+0000", 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695130+0000", 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9470000000000002e-06 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695146+0000", 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6073e-05 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695160+0000", 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3523000000000001e-05 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779407+0000", 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084247001000000002 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779545+0000", 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013839899999999999 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779681+0000", 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013551899999999999 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.171 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22493 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695222+0000", 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446921497000002, 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084642456000000005, 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.172 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22493 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695222+0000", 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695222+0000", 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695221+0000", 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695229+0000", 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7640000000000005e-06 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695248+0000", 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9513000000000001e-05 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695262+0000", 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4187e-05 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779729+0000", 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084466929999999996 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779791+0000", 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1849999999999999e-05 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779865+0000", 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3721999999999997e-05 2024-09-13T09:07:55.176 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22494 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7176192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695275+0000", 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446869093, 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084925077000000002, 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.177 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22494 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695275+0000", 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695275+0000", 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695273+0000", 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695280+0000", 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3829999999999997e-06 2024-09-13T09:07:55.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695302+0000", 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2283e-05 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695315+0000", 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3137e-05 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779928+0000", 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084612865999999995 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780066+0000", 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013803 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780200+0000", 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133745 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.181 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22495 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695327+0000", 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446816483999999, 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085037753999999993, 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22495 2024-09-13T09:07:55.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695327+0000", 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695327+0000", 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695326+0000", 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695332+0000", 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0530000000000003e-06 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695350+0000", 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7266999999999999e-05 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695363+0000", 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2876e-05 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780245+0000", 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084882231000000002 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780305+0000", 2024-09-13T09:07:55.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0693000000000003e-05 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780365+0000", 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9694000000000001e-05 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22496 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695376+0000", 2024-09-13T09:07:55.186 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446767681000001, 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085344540999999996, 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22496 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695376+0000", 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695376+0000", 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695375+0000", 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695381+0000", 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:07:55.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695399+0000", 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7286999999999999e-05 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695412+0000", 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3332999999999999e-05 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780445+0000", 2024-09-13T09:07:55.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085033009000000007 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780586+0000", 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140879 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780721+0000", 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013484 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22497 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7290880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695425+0000", 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446718421, 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085477636999999995, 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22497 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695425+0000", 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695425+0000", 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695425+0000", 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695431+0000", 2024-09-13T09:07:55.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.207e-06 2024-09-13T09:07:55.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695448+0000", 2024-09-13T09:07:55.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6875999999999999e-05 2024-09-13T09:07:55.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695461+0000", 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.347e-05 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780779+0000", 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085317492999999994 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780841+0000", 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2447999999999996e-05 2024-09-13T09:07:55.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780903+0000", 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2087000000000004e-05 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22498 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7372800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695474+0000", 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446670364999999, 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085764200999999998, 2024-09-13T09:07:55.198 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22498 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695474+0000", 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695474+0000", 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695472+0000", 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695478+0000", 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7429999999999999e-06 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695495+0000", 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7200000000000001e-05 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695508+0000", 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2924e-05 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780965+0000", 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085457003000000004 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781106+0000", 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000141015 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781238+0000", 2024-09-13T09:07:55.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013181299999999999 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22499 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695520+0000", 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446624129, 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085899188000000001, 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.203 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22499 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695520+0000", 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695520+0000", 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695519+0000", 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695525+0000", 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6830000000000002e-06 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695542+0000", 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6957e-05 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695555+0000", 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3123e-05 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781296+0000", 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085740872999999995 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781359+0000", 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2711000000000003e-05 2024-09-13T09:07:55.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781419+0000", 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9963999999999997e-05 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22500 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7454720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695567+0000", 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446577196, 2024-09-13T09:07:55.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086178355999999998, 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22500 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695567+0000", 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695567+0000", 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695566+0000", 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695572+0000", 2024-09-13T09:07:55.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2469999999999997e-06 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695588+0000", 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695601+0000", 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2799999999999999e-05 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781476+0000", 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085875192000000003 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781614+0000", 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138028 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781745+0000", 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013089300000000001 2024-09-13T09:07:55.212 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22501 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7487488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695613+0000", 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446531343, 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086327465000000006, 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.213 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22501 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695613+0000", 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695613+0000", 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695612+0000", 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695633+0000", 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.124e-05 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695651+0000", 2024-09-13T09:07:55.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8077e-05 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695664+0000", 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3186999999999999e-05 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781792+0000", 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086127924999999994 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781879+0000", 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6650000000000006e-05 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781940+0000", 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22502 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7569408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.217 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695676+0000", 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446467897000002, 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086592383999999994, 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22502 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695676+0000", 2024-09-13T09:07:55.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695676+0000", 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695675+0000", 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695679+0000", 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2300000000000002e-06 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695689+0000", 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6430000000000007e-06 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695696+0000", 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.34e-06 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781997+0000", 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086300763000000003 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782137+0000", 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013956400000000001 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782268+0000", 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013158700000000001 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.221 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22503 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695702+0000", 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.446441424, 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086760605000000005, 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22503 2024-09-13T09:07:55.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695702+0000", 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695702+0000", 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695702+0000", 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695705+0000", 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4929999999999998e-06 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695719+0000", 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3943e-05 2024-09-13T09:07:55.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695730+0000", 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1073000000000001e-05 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782319+0000", 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086588360000000003 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782399+0000", 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0536000000000003e-05 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782463+0000", 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3795999999999997e-05 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.226 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22504 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7651328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.696185+0000", 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.44595902, 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086603874999999997, 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22504 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696185+0000", 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696185+0000", 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695740+0000", 2024-09-13T09:07:55.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9995556 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696195+0000", 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00045459 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696224+0000", 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9189000000000001e-05 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696246+0000", 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1467e-05 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782516+0000", 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086270809000000004 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782653+0000", 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013632800000000001 2024-09-13T09:07:55.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782789+0000", 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136086 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22510 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7962624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.700763+0000", 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.441380604999999, 2024-09-13T09:07:55.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083641699999999999, 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22510 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700763+0000", 2024-09-13T09:07:55.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700763+0000", 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700763+0000", 2024-09-13T09:07:55.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700767+0000", 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1699999999999999e-06 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700776+0000", 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4800000000000007e-06 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700785+0000", 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2300000000000008e-06 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784119+0000", 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083334229999999995 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784263+0000", 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000144618 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784405+0000", 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014159599999999999 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22511 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.700796+0000", 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.441348039000001, 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083807406000000001, 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.236 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22511 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700796+0000", 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700796+0000", 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700795+0000", 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700800+0000", 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5169999999999999e-06 2024-09-13T09:07:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700817+0000", 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6886000000000001e-05 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700831+0000", 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3312999999999999e-05 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784449+0000", 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083618445 2024-09-13T09:07:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784511+0000", 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2385000000000007e-05 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784603+0000", 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1799999999999995e-05 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22512 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 8044544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.700842+0000", 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.441302103000002, 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084087931000000005, 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22512 2024-09-13T09:07:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700842+0000", 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700842+0000", 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700841+0000", 2024-09-13T09:07:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700847+0000", 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9739999999999999e-06 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700863+0000", 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6362999999999999e-05 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700876+0000", 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2986999999999999e-05 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784663+0000", 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083786677000000004 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784797+0000", 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013371600000000001 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784930+0000", 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013312099999999999 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:55.245 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:55.246 INFO:teuthology.orchestra.run.smithi031.stdout:1933 read 0x14c0c63 thru 0x14cf11c (0xe4ba bytes) 2024-09-13T09:07:55.246 INFO:teuthology.orchestra.run.smithi031.stdout:1936 punch from 0x1bb86a5 to 0x1bbba71, (0x33cc bytes) 2024-09-13T09:07:55.247 INFO:teuthology.orchestra.run.smithi031.stdout:1937 trunc from 0x98ecc72 to 0xd65de45 2024-09-13T09:07:55.247 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:55.247 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:55.248 INFO:teuthology.orchestra.run.smithi031.stdout:1938 trunc from 0xd65de45 to 0x621400 2024-09-13T09:07:55.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:55.248 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:55.241+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:55.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:55.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:55.249 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:55.250 INFO:teuthology.orchestra.run.smithi031.stdout:1939 write 0xba0f8d7 thru 0xba1c451 (0xcb7b bytes) 2024-09-13T09:07:55.250 INFO:teuthology.orchestra.run.smithi031.stdout:1940 write 0xcc37636 thru 0xcc3bcff (0x46ca bytes) 2024-09-13T09:07:55.250 INFO:teuthology.orchestra.run.smithi031.stdout:1941 writesame 0x251eb9e thru 0x252c9cd data_size 0x14(0xde30 bytes) 2024-09-13T09:07:55.258 INFO:teuthology.orchestra.run.smithi031.stdout:1942 read 0xafa55e0 thru 0xafaa71a (0x513b bytes) 2024-09-13T09:07:55.258 INFO:teuthology.orchestra.run.smithi031.stdout:1943 read 0x6443a4a thru 0x644e4d8 (0xaa8f bytes) 2024-09-13T09:07:55.258 INFO:teuthology.orchestra.run.smithi031.stdout:1946 punch from 0x71ce36a to 0x71ceca1, (0x937 bytes) 2024-09-13T09:07:55.272 INFO:teuthology.orchestra.run.smithi031.stdout:1947 writesame 0xde96dfe thru 0xde9fb44 data_size 0x1(0x8d47 bytes) 2024-09-13T09:07:55.273 INFO:teuthology.orchestra.run.smithi031.stdout:1948 read 0xb43ffe2 thru 0xb44abde (0xabfd bytes) 2024-09-13T09:07:55.273 INFO:teuthology.orchestra.run.smithi031.stdout:1949 read 0x84f5e41 thru 0x84f8727 (0x28e7 bytes) 2024-09-13T09:07:55.273 INFO:teuthology.orchestra.run.smithi031.stdout:1950 clone 17 order 18 su 1024 sc 14 2024-09-13T09:07:55.347 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:55.341+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:55.423 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:55.423 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:55.425 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:55.425 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:55.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:07:55.455 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:55.456 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:55.480 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:55.477+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:55.482 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:55.482 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:55.510 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:55.511 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:55.575 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:55.572+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:55.606 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:55.606 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:55.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:07:55.660 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:55.657+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:55.744 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:55.744 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:55.751 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:55.748+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:55.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:07:55.852 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:55.848+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:55.932 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:55.932 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:55.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:07:55.992 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:55.989+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:56.016 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone16 2024-09-13T09:07:56.053 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:56.048+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:56.070 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:56.070 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:56.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:07:56.172 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:56.169+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:56.251 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:56.251 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:56.254 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:56.249+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:56.261 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-13T09:07:56.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:07:56.355 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:56.352+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:56.439 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:56.439 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:56.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:07:56.456 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:56.449+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:56.557 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:56.553+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:56.572 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:56.572 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:56.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:07:56.672 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:56.669+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:56.727 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:56.741 INFO:teuthology.orchestra.run.smithi031.stdout:1953 trunc from 0xde9fb45 to 0x4541636 2024-09-13T09:07:56.756 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:56.756 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:56.758 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:56.756+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:56.772 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:07:56.786 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:56.786 INFO:teuthology.orchestra.run.smithi031.stdout:1955 read 0x676b5 thru 0x6d94c (0x6298 bytes) 2024-09-13T09:07:56.806 INFO:teuthology.orchestra.run.smithi031.stdout:1956 write 0x9d7d367 thru 0x9d89780 (0xc41a bytes) 2024-09-13T09:07:56.844 INFO:teuthology.orchestra.run.smithi031.stdout:1957 write 0x51be687 thru 0x51c85e2 (0x9f5c bytes) 2024-09-13T09:07:56.860 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:56.856+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:56.863 INFO:teuthology.orchestra.run.smithi031.stdout:1958 write 0x8fe550f thru 0x8feeeb6 (0x99a8 bytes) 2024-09-13T09:07:56.880 INFO:teuthology.orchestra.run.smithi031.stdout:1959 write 0x625f8de thru 0x62603c6 (0xae9 bytes) 2024-09-13T09:07:56.886 INFO:teuthology.orchestra.run.smithi031.stdout:1960 read 0x2e6ac4c thru 0x2e6e563 (0x3918 bytes) 2024-09-13T09:07:56.888 INFO:teuthology.orchestra.run.smithi031.stdout:1961 read 0x3f51d86 thru 0x3f5967c (0x78f7 bytes) 2024-09-13T09:07:56.888 INFO:teuthology.orchestra.run.smithi031.stdout:1963 read 0x24dc727 thru 0x24e1a7e (0x5358 bytes) 2024-09-13T09:07:56.891 INFO:teuthology.orchestra.run.smithi031.stdout:1964 read 0x895e3a6 thru 0x896af8c (0xcbe7 bytes) 2024-09-13T09:07:56.891 INFO:teuthology.orchestra.run.smithi031.stdout:1966 read 0x87e3473 thru 0x87ef45e (0xbfec bytes) 2024-09-13T09:07:56.892 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:56.892 INFO:teuthology.orchestra.run.smithi031.stdout:1968 write 0x7020308 thru 0x702e198 (0xde91 bytes) 2024-09-13T09:07:56.909 INFO:teuthology.orchestra.run.smithi031.stdout:1970 writesame 0x1795560 thru 0x17969a1 data_size 0x1(0x1442 bytes) 2024-09-13T09:07:56.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:07:56.956 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:56.956 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:56.961 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:56.956+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:56.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:07:57.071 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:57.068+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:57.087 INFO:teuthology.orchestra.run.smithi031.stdout:1971 writesame 0x2a654a9 thru 0x2a6f45b data_size 0x1(0x9fb3 bytes) 2024-09-13T09:07:57.163 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:57.160+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:57.198 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:57.198 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:57.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:07:57.265 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:57.261+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:57.349 INFO:teuthology.orchestra.run.smithi031.stdout:1972 read 0x5d28ffa thru 0x5d2f09c (0x60a3 bytes) 2024-09-13T09:07:57.349 INFO:teuthology.orchestra.run.smithi031.stdout:1973 punch from 0x7a44edd to 0x7a54969, (0xfa8c bytes) 2024-09-13T09:07:57.356 INFO:teuthology.orchestra.run.smithi031.stdout:1974 write 0xa27ccb6 thru 0xa28776c (0xaab7 bytes) 2024-09-13T09:07:57.365 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:57.361+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:57.388 INFO:teuthology.orchestra.run.smithi031.stdout:1976 write 0xdb1c1e3 thru 0xdb283c9 (0xc1e7 bytes) 2024-09-13T09:07:57.412 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:57.417 INFO:teuthology.orchestra.run.smithi031.stdout:1978 write 0xdf37ae7 thru 0xdf3fe46 (0x8360 bytes) 2024-09-13T09:07:57.436 INFO:teuthology.orchestra.run.smithi031.stdout:1979 punch from 0x327c2c9 to 0x327d81c, (0x1553 bytes) 2024-09-13T09:07:57.436 INFO:teuthology.orchestra.run.smithi031.stdout:1980 write 0x9ca0207 thru 0x9ca85e1 (0x83db bytes) 2024-09-13T09:07:57.442 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:57.442 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:57.450 INFO:teuthology.orchestra.run.smithi031.stdout:1982 writesame 0xeb4fccd thru 0xeb5a51c data_size 0x8(0xa850 bytes) 2024-09-13T09:07:57.450 INFO:teuthology.orchestra.run.smithi031.stdout:1983 writesame 0x39303 thru 0x46e91 data_size 0x1(0xdb8f bytes) 2024-09-13T09:07:57.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:07:57.466 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:57.461+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:57.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:07:57.579 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:57.576+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:57.620 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:07:57.621 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:07:57.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:07:57.668 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:57.661+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:57.697 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:07:57.697 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:07:57.728 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:07:57.728 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:07:57.759 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:07:57.759 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:07:57.768 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:57.765+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:57.789 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:07:57.789 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:07:57.825 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:07:57.825 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:07:57.843 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:57.843 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:57.858 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:07:57.858 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:07:57.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:07:57.869 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:57.865+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:57.882 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:07:57.882 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:07:57.892 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:07:57.916 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:07:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:07:57.970 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:57.965+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:57.980 INFO:teuthology.orchestra.run.smithi031.stdout:1984 punch from 0xe1b0d52 to 0xe1bee6f, (0xe11d bytes) 2024-09-13T09:07:57.980 INFO:teuthology.orchestra.run.smithi031.stdout:1986 writesame 0xc55cf96 thru 0xc55dd04 data_size 0x1(0xd6f bytes) 2024-09-13T09:07:57.989 INFO:teuthology.orchestra.run.smithi031.stdout:1987 read 0xa430b8b thru 0xa43a7b5 (0x9c2b bytes) 2024-09-13T09:07:58.000 INFO:teuthology.orchestra.run.smithi031.stdout:1989 trunc from 0xeb5a51d to 0xaef5a7b 2024-09-13T09:07:58.019 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.324223875999998, 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01204616, 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 35 2024-09-13T09:07:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694090+0000", 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694089+0000", 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694329+0000", 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000240051 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694360+0000", 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0490000000000001e-05 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694377+0000", 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7640000000000001e-05 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701403+0000", 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070253590000000001 2024-09-13T09:07:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.701682+0000", 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027976800000000001 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.702044+0000", 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036119 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705969+0000", 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039254570000000003 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705984+0000", 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4396000000000001e-05 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706063+0000", 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9839000000000006e-05 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706137+0000", 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3176999999999994e-05 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.323915862, 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012010730000000001, 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 36 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694398+0000", 2024-09-13T09:07:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694397+0000", 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694409+0000", 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2277e-05 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694449+0000", 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9533000000000002e-05 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694465+0000", 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.56e-05 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703386+0000", 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0089211740000000005 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703688+0000", 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030240099999999999 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.703902+0000", 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213525 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706154+0000", 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002252783 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706233+0000", 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8169000000000003e-05 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706306+0000", 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3232999999999995e-05 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706409+0000", 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103259 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.323827948999998, 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012438360000000001, 2024-09-13T09:07:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694486+0000", 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694485+0000", 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694497+0000", 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694518+0000", 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0415999999999999e-05 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.694545+0000", 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6752999999999999e-05 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704569+0000", 2024-09-13T09:07:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010024099 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.704863+0000", 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029461100000000002 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.705120+0000", 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025678499999999999 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706437+0000", 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001317127 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706757+0000", 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031999099999999999 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706830+0000", 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3126000000000004e-05 2024-09-13T09:07:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.706925+0000", 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4399000000000002e-05 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.053303587999999, 2024-09-13T09:07:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013291214000000001, 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 110 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965011+0000", 2024-09-13T09:07:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965010+0000", 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965017+0000", 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4270000000000002e-06 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965057+0000", 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9351999999999999e-05 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965071+0000", 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4579999999999999e-05 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973277+0000", 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082060109999999992 2024-09-13T09:07:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973401+0000", 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000123342 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.973603+0000", 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000201941 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978035+0000", 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044325060000000001 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978132+0000", 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7312999999999995e-05 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978206+0000", 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3986000000000006e-05 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978302+0000", 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5508999999999996e-05 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.053215825000002, 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013499359000000001, 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 111 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965098+0000", 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965097+0000", 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965107+0000", 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969999999999995e-06 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965121+0000", 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4056000000000001e-05 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965139+0000", 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8317000000000001e-05 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974658+0000", 2024-09-13T09:07:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0095193959999999994 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974786+0000", 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127212 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.974978+0000", 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019252599999999999 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978415+0000", 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034365160000000001 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978438+0000", 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3504000000000001e-05 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978528+0000", 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9988999999999995e-05 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978598+0000", 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9715999999999995e-05 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.053163771999998, 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014714316, 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 112 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965150+0000", 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965297+0000", 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000147312 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965312+0000", 2024-09-13T09:07:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4846e-05 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965325+0000", 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2773e-05 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.975868+0000", 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010542925999999999 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976011+0000", 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014285200000000001 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.976204+0000", 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019317499999999999 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978619+0000", 2024-09-13T09:07:58.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024152539999999999 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979722+0000", 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001103099 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979802+0000", 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9689000000000002e-05 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979865+0000", 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3003000000000006e-05 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.052978690000003, 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014775662, 2024-09-13T09:07:58.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 113 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965336+0000", 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965335+0000", 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965343+0000", 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999992e-06 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965390+0000", 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7367000000000002e-05 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965400+0000", 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0332999999999999e-05 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977169+0000", 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011769109 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977305+0000", 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135759 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.977501+0000", 2024-09-13T09:07:58.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019627499999999999 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979893+0000", 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002392061 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979982+0000", 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8337000000000005e-05 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980050+0000", 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8169000000000004e-05 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980111+0000", 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1276000000000001e-05 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.052904200999997, 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017235837, 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965410+0000", 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965409+0000", 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:58.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965416+0000", 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3400000000000003e-06 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965427+0000", 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1387e-05 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965438+0000", 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0866000000000001e-05 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978698+0000", 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013259918000000001 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.978826+0000", 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012871599999999999 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.979019+0000", 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019267399999999999 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982472+0000", 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034531940000000001 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982501+0000", 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8914000000000001e-05 2024-09-13T09:07:58.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982580+0000", 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8374999999999994e-05 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982646+0000", 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6272999999999994e-05 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.052849881, 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018283074999999999, 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965464+0000", 2024-09-13T09:07:58.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965603+0000", 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139472 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965633+0000", 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0417e-05 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965646+0000", 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2543e-05 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980226+0000", 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014580437 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980351+0000", 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012426900000000001 2024-09-13T09:07:58.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.980546+0000", 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000195546 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982674+0000", 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021279900000000002 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983607+0000", 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093225899999999995 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983681+0000", 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4740000000000006e-05 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983747+0000", 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6159000000000001e-05 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.052651795999999, 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019556396, 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 116 2024-09-13T09:07:58.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965662+0000", 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965670+0000", 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7870000000000002e-06 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965694+0000", 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4635999999999999e-05 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965711+0000", 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7303e-05 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981488+0000", 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015776748 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981620+0000", 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013168199999999999 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.981817+0000", 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019701899999999999 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984005+0000", 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021882529999999998 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985091+0000", 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010853950000000001 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985159+0000", 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8139000000000006e-05 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985219+0000", 2024-09-13T09:07:58.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0136999999999997e-05 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.052570533000001, 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019824747, 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 117 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965744+0000", 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965743+0000", 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965750+0000", 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.25e-06 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965774+0000", 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3770000000000001e-05 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.965784+0000", 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6339999999999996e-06 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982722+0000", 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016938482000000001 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.982855+0000", 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013241199999999999 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.983118+0000", 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026305499999999999 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.984622+0000", 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001503852 2024-09-13T09:07:58.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985444+0000", 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082297999999999996 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985511+0000", 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6645999999999999e-05 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:56.985568+0000", 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.732e-05 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.558275817999998, 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014366419, 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 137 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460038+0000", 2024-09-13T09:07:58.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460037+0000", 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460292+0000", 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025541799999999999 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460317+0000", 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4392999999999998e-05 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460334+0000", 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7436999999999999e-05 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469354+0000", 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0090198299999999995 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469473+0000", 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118983 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.469664+0000", 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191378 2024-09-13T09:07:58.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472724+0000", 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030597430000000002 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474242+0000", 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015184350000000001 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474350+0000", 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010723200000000001 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474405+0000", 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5179999999999997e-05 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.095 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.557960180999999, 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015873761, 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 138 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460354+0000", 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460353+0000", 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460364+0000", 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.152e-05 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460384+0000", 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9179999999999999e-05 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460398+0000", 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4283e-05 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471713+0000", 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011314706000000001 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.471842+0000", 2024-09-13T09:07:58.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012978299999999999 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472053+0000", 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000210261 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476066+0000", 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040129190000000002 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476090+0000", 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4689999999999999e-05 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476172+0000", 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1689999999999999e-05 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476228+0000", 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5939000000000001e-05 2024-09-13T09:07:58.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.557899851000002, 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016977385000000001, 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 139 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460414+0000", 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460413+0000", 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460424+0000", 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.076e-05 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460442+0000", 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7893e-05 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460456+0000", 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4197e-05 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472812+0000", 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012355560999999999 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.472933+0000", 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012084200000000001 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.473697+0000", 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00076427799999999996 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476330+0000", 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026328189999999998 2024-09-13T09:07:58.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477269+0000", 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093877999999999995 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477337+0000", 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8423000000000005e-05 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477392+0000", 2024-09-13T09:07:58.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4731999999999997e-05 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.557841945, 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017258564000000001, 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 140 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460472+0000", 2024-09-13T09:07:58.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460471+0000", 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460709+0000", 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000237288 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460738+0000", 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9876e-05 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460755+0000", 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6229999999999999e-05 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474490+0000", 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013735044 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474631+0000", 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014173199999999999 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.474821+0000", 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000189216 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477405+0000", 2024-09-13T09:07:58.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025845500000000001 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477610+0000", 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020505499999999999 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477682+0000", 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1486e-05 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477731+0000", 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9110000000000002e-05 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4207.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.557539347999999, 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017377179999999999, 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4207", 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.60:0/2157845931", 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 141 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460775+0000", 2024-09-13T09:07:58.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460774+0000", 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460801+0000", 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7246999999999998e-05 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460824+0000", 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3336000000000002e-05 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.460841+0000", 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7110000000000001e-05 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476381+0000", 2024-09-13T09:07:58.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015539814000000001 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476511+0000", 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012941499999999999 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.476704+0000", 2024-09-13T09:07:58.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019368900000000001 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.477997+0000", 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012926700000000001 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478026+0000", 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8713000000000001e-05 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478095+0000", 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9313000000000005e-05 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:06:57.478152+0000", 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7166000000000001e-05 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7649 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:58.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.164974180999998, 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013081786, 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7649 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:58.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853340+0000", 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853339+0000", 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853345+0000", 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7839999999999998e-06 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853365+0000", 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9054000000000001e-05 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.853374+0000", 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6479999999999998e-06 2024-09-13T09:07:58.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858418+0000", 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050442869999999997 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858525+0000", 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010602099999999999 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.858992+0000", 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046758299999999999 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.861712+0000", 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027195700000000001 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866315+0000", 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046037439999999999 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866381+0000", 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5121000000000002e-05 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.866422+0000", 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1331000000000001e-05 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7657 3.10 3:08eae27d:::rbd_data.11242ae5e73a.000000000000000b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 13762560~1572864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.150181887000002, 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018927840000000001, 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7657 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868132+0000", 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.868131+0000", 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:07:58.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870832+0000", 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027013660000000002 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870853+0000", 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.107e-05 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870862+0000", 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7830000000000004e-06 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870910+0000", 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871083+0000", 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017277500000000001 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.871386+0000", 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030378600000000001 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878951+0000", 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075642649999999997 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.886982+0000", 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080316160000000001 2024-09-13T09:07:58.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887025+0000", 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3288999999999998e-05 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887060+0000", 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4706999999999998e-05 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:7660 3.10 3:09f93e87:::rbd_object_map.11242ae5e73a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.147438798000003, 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016257230000000001, 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7660 2024-09-13T09:07:58.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870875+0000", 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870874+0000", 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870879+0000", 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5669999999999996e-06 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870885+0000", 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0260000000000004e-06 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.870888+0000", 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9749999999999999e-06 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876842+0000", 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0059541780000000001 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.876952+0000", 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109577 2024-09-13T09:07:58.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.877351+0000", 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00039971899999999999 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.878995+0000", 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001643179 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887073+0000", 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080780109999999995 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887100+0000", 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7585000000000001e-05 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:24.887133+0000", 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2533000000000002e-05 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0657877009999996, 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:07:58.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:07:58.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:07:58.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:07:58.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.146 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:58.147 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:58.147 INFO:teuthology.orchestra.run.smithi031.stdout:1991 writesame 0x49c24ff thru 0x49d189e data_size 0x10(0xf3a0 bytes) 2024-09-13T09:07:58.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:07:58.148 INFO:teuthology.orchestra.run.smithi031.stdout:1993 writesame 0x6796328 thru 0x67a278a data_size 0x5(0xc463 bytes) 2024-09-13T09:07:58.149 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:07:58.149 INFO:teuthology.orchestra.run.smithi031.stdout:1995 trunc from 0xaef5a7b to 0x15920ed 2024-09-13T09:07:58.149 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:58.141+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:58.248 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:58.241+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:58.294 INFO:teuthology.orchestra.run.smithi031.stdout:1998 clone 18 order 25 su 524288 sc 4 2024-09-13T09:07:58.328 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:58.328 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:58.329 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:58.329 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:58.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:07:58.390 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:58.388+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:58.449 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:07:58.444+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:07:58.455 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:58.455 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:58.456 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:58.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:07:58.555 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:58.552+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:58.625 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.284085366999999, 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:07:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:07:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.284068594000001, 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:07:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:07:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.284031289000001, 2024-09-13T09:07:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:07:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.284011575000001, 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.28399525, 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283979383999998, 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:07:58.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:07:58.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283963599, 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:07:58.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283944527999999, 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283927981000001, 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:07:58.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283900381999999, 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:07:58.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.28388314, 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283852697, 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:07:58.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:07:58.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283833781999999, 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:07:58.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:07:58.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283815994000001, 2024-09-13T09:07:58.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:07:58.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.28379077, 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:07:58.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283772566, 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:07:58.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.700 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283752247999999, 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:58.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:07:58.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:07:58.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283733699999999, 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:58.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:07:58.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283716426000002, 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.283698972, 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:58.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:07:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.718 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:58.719 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:58.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:07:58.721 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:58.717+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:58.821 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:58.816+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:58.887 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:58.887 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:58.888 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:07:58.888 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:58.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:07:58.922 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:07:58.917+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:07:58.999 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone17 intact 2024-09-13T09:07:59.023 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:07:59.017+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:07:59.068 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:07:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:07:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:07:59.068 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:59.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:07:59.083 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-13T09:07:59.124 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:59.121+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:59.200 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:07:59.200 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:07:59.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:07:59.200 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.068773659000001, 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:07:59.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:07:59.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:07:59.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:07:59.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14831 3.14 3:2fa7af23:::rbd_data.1124ea59ddac.000000000000001c:a [read 409600~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.962965646000001, 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071499264000000007, 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14831 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:59.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236539+0000", 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236541+0000", 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3910000000000001e-06 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236548+0000", 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.236553+0000", 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1449999999999999e-06 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307720+0000", 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071166574999999996 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.307883+0000", 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000163809 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308039+0000", 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155229 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14835 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1916928~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961607753999999, 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071108525000000006, 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14835 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237897+0000", 2024-09-13T09:07:59.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237908+0000", 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1191e-05 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237929+0000", 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1189e-05 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237936+0000", 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308827+0000", 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.070891320999999993 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.308924+0000", 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6665999999999999e-05 2024-09-13T09:07:59.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309006+0000", 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2231000000000006e-05 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14836 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961562777000001, 2024-09-13T09:07:59.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071290567999999999, 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14836 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237942+0000", 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237945+0000", 2024-09-13T09:07:59.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6229999999999999e-06 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237953+0000", 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237958+0000", 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0309999999999998e-06 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309041+0000", 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071083576999999995 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309132+0000", 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0500000000000004e-05 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309233+0000", 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101116 2024-09-13T09:07:59.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14837 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3096576~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961542971, 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071483336999999994, 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14837 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237962+0000", 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237964+0000", 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237971+0000", 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237975+0000", 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6360000000000002e-06 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309280+0000", 2024-09-13T09:07:59.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071304765000000006 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309359+0000", 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9066999999999994e-05 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309445+0000", 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6255999999999995e-05 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14838 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3686400~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961523380999999, 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071691981000000002, 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14838 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237982+0000", 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237981+0000", 2024-09-13T09:07:59.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237984+0000", 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6860000000000002e-06 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237990+0000", 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9920000000000004e-06 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237995+0000", 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5709999999999999e-06 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309487+0000", 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071492752000000007 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309575+0000", 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7106000000000003e-05 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309674+0000", 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9208000000000003e-05 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14844 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 163840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961505947999999, 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072094264000000005, 2024-09-13T09:07:59.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14844 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.237999+0000", 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238002+0000", 2024-09-13T09:07:59.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8289999999999999e-06 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238008+0000", 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.032e-06 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238013+0000", 2024-09-13T09:07:59.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1340000000000003e-06 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309739+0000", 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.071726390000000001 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.309786+0000", 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6561999999999999e-05 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310093+0000", 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030766000000000002 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14845 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 753664~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961487786999999, 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072306747000000005, 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14845 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238017+0000", 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238019+0000", 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4449999999999999e-06 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238025+0000", 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4870000000000002e-06 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238029+0000", 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3769999999999998e-06 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310139+0000", 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072110056000000006 2024-09-13T09:07:59.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310224+0000", 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4291999999999999e-05 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310324+0000", 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100405 2024-09-13T09:07:59.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14846 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1343488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961470023, 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072637322000000004, 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14846 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238035+0000", 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238037+0000", 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-09-13T09:07:59.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238045+0000", 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0960000000000001e-06 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238051+0000", 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0929999999999999e-06 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310365+0000", 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072313575000000005 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310432+0000", 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6888000000000001e-05 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310672+0000", 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024050699999999999 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14847 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1933312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961448892, 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072846102999999995, 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14847 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238056+0000", 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238058+0000", 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5119999999999998e-06 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238063+0000", 2024-09-13T09:07:59.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238067+0000", 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4750000000000004e-06 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310717+0000", 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072649243000000002 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310803+0000", 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6147e-05 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310902+0000", 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9474000000000003e-05 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14848 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 2523136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:59.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.961433261, 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072985396999999994, 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14848 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238072+0000", 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238074+0000", 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.227e-06 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238080+0000", 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1639999999999998e-06 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238085+0000", 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1410000000000004e-06 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310939+0000", 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072853862000000005 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.310997+0000", 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8248000000000002e-05 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311057+0000", 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0071999999999997e-05 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14849 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:59.255 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.960498766000001, 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072287373000000002, 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14849 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:59.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239006+0000", 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.238089+0000", 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9990826 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239012+0000", 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092334199999999996 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239026+0000", 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3351e-05 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239034+0000", 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4419999999999998e-06 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311104+0000", 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072069464 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311194+0000", 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9922999999999999e-05 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311294+0000", 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001002 2024-09-13T09:07:59.259 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14850 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 3702784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.960462804999999, 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072393710999999999, 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.260 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14850 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:59.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239042+0000", 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239045+0000", 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4429999999999999e-06 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239053+0000", 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5179999999999998e-06 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239058+0000", 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2560000000000002e-06 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311330+0000", 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072271613999999998 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311381+0000", 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0908000000000001e-05 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311436+0000", 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5513000000000002e-05 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14861 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 180224~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.960239858000001, 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072582646000000001, 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14861 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239265+0000", 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239270+0000", 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1039999999999998e-06 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239280+0000", 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0210999999999999e-05 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239285+0000", 2024-09-13T09:07:59.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.967e-06 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311481+0000", 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072196249000000004 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311573+0000", 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1588999999999993e-05 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311848+0000", 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027526699999999999 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14862 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 327680~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.960216547999998, 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072869149999999994, 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14862 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239289+0000", 2024-09-13T09:07:59.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239288+0000", 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239295+0000", 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2320000000000004e-06 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239302+0000", 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.793e-06 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239305+0000", 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9380000000000001e-06 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311878+0000", 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072573031999999996 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.311923+0000", 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4351000000000003e-05 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312158+0000", 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023512399999999999 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14863 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 770048~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.960197283999999, 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072990372999999997, 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14863 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239308+0000", 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239309+0000", 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3939999999999999e-06 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239312+0000", 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239315+0000", 2024-09-13T09:07:59.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7630000000000001e-06 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312186+0000", 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072871378000000001 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312243+0000", 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6487000000000003e-05 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312298+0000", 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5506e-05 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14864 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 917504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.960137474, 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073255022000000003, 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14864 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239368+0000", 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239317+0000", 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999949 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239372+0000", 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5457000000000001e-05 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239381+0000", 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.003e-06 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239387+0000", 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6130000000000003e-06 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312327+0000", 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072940489999999997 2024-09-13T09:07:59.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312409+0000", 2024-09-13T09:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.153e-05 2024-09-13T09:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312623+0000", 2024-09-13T09:07:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000213799 2024-09-13T09:07:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14865 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.284 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.959866949999999, 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073307136999999994, 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14865 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239638+0000", 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239390+0000", 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999752 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239643+0000", 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025296200000000002 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239653+0000", 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7839999999999998e-06 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239660+0000", 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0570000000000004e-06 2024-09-13T09:07:59.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312655+0000", 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.072994857999999996 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312731+0000", 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5754999999999996e-05 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312945+0000", 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00021457799999999999 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:14866 3.14 3:2d8be43f:::rbd_data.1124fee72b32.0000000000000003:9 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.959838779999998, 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073614097000000003, 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.289 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14866 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239666+0000", 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239669+0000", 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.858e-06 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239675+0000", 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6989999999999997e-06 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.239680+0000", 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:07:59.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.312975+0000", 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.073294697000000006 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313051+0000", 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6297999999999994e-05 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:31.313280+0000", 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022920800000000001 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.2404914790000001, 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:07:59.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:07:59.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:07:59.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:07:59.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:07:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:07:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:07:59.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:07:59.299 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:07:59.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:07:59.301 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:07:59.296+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:07:59.402 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:07:59.396+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:07:59.429 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:59.429 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:59.429 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:07:59.429 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:59.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:07:59.503 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:59.497+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:59.509 INFO:teuthology.orchestra.run.smithi031.stdout:2000 trunc from 0x15920ed to 0x2f2bc46 2024-09-13T09:07:59.548 INFO:teuthology.orchestra.run.smithi031.stdout:2002 write 0x9cf31e8 thru 0x9cf9f3e (0x6d57 bytes) 2024-09-13T09:07:59.555 INFO:teuthology.orchestra.run.smithi031.stdout:2003 read 0x93c082c thru 0x93c9f7f (0x9754 bytes) 2024-09-13T09:07:59.555 INFO:teuthology.orchestra.run.smithi031.stdout:2007 read 0x639dd2 thru 0x6414d3 (0x7702 bytes) 2024-09-13T09:07:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:2010 read 0x514f3da thru 0x51514b8 (0x20df bytes) 2024-09-13T09:07:59.559 INFO:teuthology.orchestra.run.smithi031.stdout:2011 write 0x374265e thru 0x3746bbd (0x4560 bytes) 2024-09-13T09:07:59.577 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:59.577 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:07:59.577 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:07:59.577 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:07:59.577 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:59.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:07:59.633 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:59.629+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:07:59.705 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:07:59.700+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:07:59.757 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:07:59.758 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:07:59.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:07:59.758 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:07:59.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.758 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22488 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 6864896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.758 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695014+0000", 2024-09-13T09:07:59.758 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057790717, 2024-09-13T09:07:59.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083619160999999997, 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22488 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695014+0000", 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695014+0000", 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695013+0000", 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695017+0000", 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4359999999999998e-06 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695025+0000", 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4940000000000003e-06 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695032+0000", 2024-09-13T09:07:59.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9890000000000003e-06 2024-09-13T09:07:59.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778364+0000", 2024-09-13T09:07:59.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083332099000000007 2024-09-13T09:07:59.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778502+0000", 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013811300000000001 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778633+0000", 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130646 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22489 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695039+0000", 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.05776603, 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083795781999999999, 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22489 2024-09-13T09:07:59.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695039+0000", 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695039+0000", 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695038+0000", 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695050+0000", 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1769e-05 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695059+0000", 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1640000000000007e-06 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695066+0000", 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0729999999999999e-06 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778681+0000", 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083614843999999994 2024-09-13T09:07:59.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778742+0000", 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1271000000000003e-05 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778834+0000", 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2270999999999998e-05 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22490 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 6979584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695073+0000", 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057732088000002, 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084084148999999997, 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22490 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695073+0000", 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695073+0000", 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695072+0000", 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695075+0000", 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4300000000000002e-06 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695084+0000", 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6570000000000003e-06 2024-09-13T09:07:59.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695091+0000", 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8170000000000001e-06 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.778888+0000", 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.08379702 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779022+0000", 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013453799999999999 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779157+0000", 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439699999999999 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22491 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695097+0000", 2024-09-13T09:07:59.773 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057707908000001, 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084251518999999997, 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22491 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695097+0000", 2024-09-13T09:07:59.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695097+0000", 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695096+0000", 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695100+0000", 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4769999999999999e-06 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695108+0000", 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5029999999999997e-06 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695115+0000", 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9800000000000001e-06 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779204+0000", 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084088642000000005 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779264+0000", 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0210999999999997e-05 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779348+0000", 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4308999999999995e-05 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22492 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7061504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695126+0000", 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057678473999999, 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084554658000000005, 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22492 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695126+0000", 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695126+0000", 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695125+0000", 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695130+0000", 2024-09-13T09:07:59.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9470000000000002e-06 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695146+0000", 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6073e-05 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695160+0000", 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3523000000000001e-05 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779407+0000", 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084247001000000002 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779545+0000", 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013839899999999999 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779681+0000", 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013551899999999999 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.782 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22493 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695222+0000", 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057582209, 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084642456000000005, 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22493 2024-09-13T09:07:59.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695222+0000", 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695222+0000", 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695221+0000", 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695229+0000", 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7640000000000005e-06 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695248+0000", 2024-09-13T09:07:59.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9513000000000001e-05 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695262+0000", 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4187e-05 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779729+0000", 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084466929999999996 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779791+0000", 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1849999999999999e-05 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779865+0000", 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3721999999999997e-05 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22494 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7176192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695275+0000", 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057529805000001, 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084925077000000002, 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22494 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.788 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695275+0000", 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695275+0000", 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695273+0000", 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695280+0000", 2024-09-13T09:07:59.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3829999999999997e-06 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695302+0000", 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2283e-05 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695315+0000", 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3137e-05 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.779928+0000", 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084612865999999995 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780066+0000", 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013803 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780200+0000", 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133745 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.792 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.793 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.793 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22495 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7208960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.793 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695327+0000", 2024-09-13T09:07:59.793 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057477196000001, 2024-09-13T09:07:59.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085037753999999993, 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22495 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695327+0000", 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695327+0000", 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695326+0000", 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695332+0000", 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0530000000000003e-06 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695350+0000", 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7266999999999999e-05 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695363+0000", 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2876e-05 2024-09-13T09:07:59.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780245+0000", 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084882231000000002 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780305+0000", 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0693000000000003e-05 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780365+0000", 2024-09-13T09:07:59.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9694000000000001e-05 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22496 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695376+0000", 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057428392999999, 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085344540999999996, 2024-09-13T09:07:59.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22496 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695376+0000", 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695376+0000", 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695375+0000", 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695381+0000", 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695399+0000", 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7286999999999999e-05 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695412+0000", 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3332999999999999e-05 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780445+0000", 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085033009000000007 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780586+0000", 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140879 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780721+0000", 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013484 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22497 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7290880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.803 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695425+0000", 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057379133000001, 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085477636999999995, 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22497 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.804 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695425+0000", 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695425+0000", 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695425+0000", 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695431+0000", 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.207e-06 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695448+0000", 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6875999999999999e-05 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695461+0000", 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.347e-05 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780779+0000", 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085317492999999994 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780841+0000", 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2447999999999996e-05 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780903+0000", 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2087000000000004e-05 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.808 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22498 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7372800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695474+0000", 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057331077000001, 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085764200999999998, 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22498 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.809 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695474+0000", 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695474+0000", 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695472+0000", 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695478+0000", 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7429999999999999e-06 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695495+0000", 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7200000000000001e-05 2024-09-13T09:07:59.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695508+0000", 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2924e-05 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.780965+0000", 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085457003000000004 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781106+0000", 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000141015 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781238+0000", 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013181299999999999 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22499 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7405568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695520+0000", 2024-09-13T09:07:59.813 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057284841000001, 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085899188000000001, 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22499 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695520+0000", 2024-09-13T09:07:59.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695520+0000", 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695519+0000", 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695525+0000", 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6830000000000002e-06 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695542+0000", 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6957e-05 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695555+0000", 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3123e-05 2024-09-13T09:07:59.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781296+0000", 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085740872999999995 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781359+0000", 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2711000000000003e-05 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781419+0000", 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9963999999999997e-05 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22500 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7454720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695567+0000", 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057237908000001, 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086178355999999998, 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.818 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22500 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695567+0000", 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695567+0000", 2024-09-13T09:07:59.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695566+0000", 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695572+0000", 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2469999999999997e-06 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695588+0000", 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695601+0000", 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2799999999999999e-05 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781476+0000", 2024-09-13T09:07:59.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085875192000000003 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781614+0000", 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138028 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781745+0000", 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013089300000000001 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22501 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7487488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695613+0000", 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057192055000002, 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086327465000000006, 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22501 2024-09-13T09:07:59.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695613+0000", 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695613+0000", 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695612+0000", 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695633+0000", 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.124e-05 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695651+0000", 2024-09-13T09:07:59.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8077e-05 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695664+0000", 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3186999999999999e-05 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781792+0000", 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086127924999999994 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781879+0000", 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6650000000000006e-05 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781940+0000", 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1372999999999997e-05 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22502 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7569408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695676+0000", 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057128608999999, 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086592383999999994, 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22502 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695676+0000", 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695676+0000", 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695675+0000", 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695679+0000", 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2300000000000002e-06 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695689+0000", 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6430000000000007e-06 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695696+0000", 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.34e-06 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.781997+0000", 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086300763000000003 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782137+0000", 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013956400000000001 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782268+0000", 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013158700000000001 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22503 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7602176~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.695702+0000", 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.057102136000001, 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086760605000000005, 2024-09-13T09:07:59.832 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22503 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695702+0000", 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695702+0000", 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695702+0000", 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695705+0000", 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4929999999999998e-06 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695719+0000", 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3943e-05 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695730+0000", 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1073000000000001e-05 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782319+0000", 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086588360000000003 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782399+0000", 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0536000000000003e-05 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782463+0000", 2024-09-13T09:07:59.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3795999999999997e-05 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22504 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7651328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.696185+0000", 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.056619732000001, 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086603874999999997, 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.837 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22504 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696185+0000", 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696185+0000", 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.695740+0000", 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9995556 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696195+0000", 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00045459 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696224+0000", 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9189000000000001e-05 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.696246+0000", 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1467e-05 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782516+0000", 2024-09-13T09:07:59.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086270809000000004 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782653+0000", 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013632800000000001 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.782789+0000", 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136086 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22510 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7962624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.700763+0000", 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.052041317, 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083641699999999999, 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22510 2024-09-13T09:07:59.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700763+0000", 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700763+0000", 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700763+0000", 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700767+0000", 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1699999999999999e-06 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700776+0000", 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4800000000000007e-06 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700785+0000", 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2300000000000008e-06 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784119+0000", 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083334229999999995 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784263+0000", 2024-09-13T09:07:59.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000144618 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784405+0000", 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014159599999999999 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22511 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.700796+0000", 2024-09-13T09:07:59.846 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.052008750999999, 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083807406000000001, 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22511 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700796+0000", 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700796+0000", 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700795+0000", 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700800+0000", 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5169999999999999e-06 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700817+0000", 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6886000000000001e-05 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700831+0000", 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3312999999999999e-05 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784449+0000", 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083618445 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784511+0000", 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2385000000000007e-05 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784603+0000", 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1799999999999995e-05 2024-09-13T09:07:59.850 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:22512 3.1b 3:dcaf816f:::rbd_data.1124d0fab35.0000000000000001:c [read 8044544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.700842+0000", 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 24.051962815, 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084087931000000005, 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:07:59.851 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22512 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700842+0000", 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700842+0000", 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:07:59.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700841+0000", 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700847+0000", 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9739999999999999e-06 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700863+0000", 2024-09-13T09:07:59.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6362999999999999e-05 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.700876+0000", 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2986999999999999e-05 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784663+0000", 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083786677000000004 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:07:59.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784797+0000", 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013371600000000001 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.784930+0000", 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013312099999999999 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:07:59.855 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:07:59.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:07:59.858 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:07:59.853+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:07:59.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:07:59.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:07:59.958 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:07:59.953+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:00.028 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:08:00.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:08:00.036 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:00.036 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:00.036 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:00.036 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:00.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:00.087 INFO:teuthology.orchestra.run.smithi031.stdout:2012 write 0x229f42f thru 0x22ab499 (0xc06b bytes) 2024-09-13T09:08:00.092 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:00.089+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:00.186 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:00.168+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:00.189 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:00.189 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:00.248 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:00.249 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:00.252 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:00.252 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:00.252 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:00.252 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:00.252 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:00.261 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:00.256+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:00.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:00.309 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:00.309 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:00.345 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:00.346 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:00.361 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:00.357+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:00.393 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:00.432 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:00.433 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:00.433 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:00.433 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:00.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:00.465 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:00.466 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:00.497 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:00.493+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:00.563 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:00.560+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:00.619 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:00.619 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:00.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:00.681 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:00.677+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:00.691 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:00.699 INFO:teuthology.orchestra.run.smithi031.stdout:2014 writesame 0xa8d5411 thru 0xa8d9be9 data_size 0x1(0x47d9 bytes) 2024-09-13T09:08:00.701 INFO:teuthology.orchestra.run.smithi031.stdout:2016 trunc from 0xa8d9bea to 0xa0b4d82 2024-09-13T09:08:00.704 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:00.704 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:00.709 INFO:teuthology.orchestra.run.smithi031.stdout:2017 writesame 0x76c42ae thru 0x76c78e4 data_size 0x1(0x3637 bytes) 2024-09-13T09:08:00.720 INFO:teuthology.orchestra.run.smithi031.stdout:2021 read 0x8f882cb thru 0x8f8d192 (0x4ec8 bytes) 2024-09-13T09:08:00.720 INFO:teuthology.orchestra.run.smithi031.stdout:2022 read 0x2789b0f thru 0x278d103 (0x35f5 bytes) 2024-09-13T09:08:00.720 INFO:teuthology.orchestra.run.smithi031.stdout:2025 read 0x57888a9 thru 0x5795924 (0xd07c bytes) 2024-09-13T09:08:00.720 INFO:teuthology.orchestra.run.smithi031.stdout:2026 punch from 0x6fe3175 to 0x6febfe1, (0x8e6c bytes) 2024-09-13T09:08:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:2029 trunc from 0xa0b4d82 to 0x9867210 2024-09-13T09:08:00.731 INFO:teuthology.orchestra.run.smithi031.stdout:2030 punch from 0x8cd29ae to 0x8cd41c2, (0x1814 bytes) 2024-09-13T09:08:00.731 INFO:teuthology.orchestra.run.smithi031.stdout:2031 punch from 0x16ed73b to 0x16f9e57, (0xc71c bytes) 2024-09-13T09:08:00.731 INFO:teuthology.orchestra.run.smithi031.stdout:2033 write 0x219f074 thru 0x21a7ebc (0x8e49 bytes) 2024-09-13T09:08:00.764 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:00.761+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:00.779 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:00.779 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:00.795 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:00.881 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:00.877+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:00.915 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:08:00.918 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:08:00.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:08:00.968 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:00.969 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:00.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:00.995 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:00.992+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:01.066 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:01.060+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:01.143 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:01.143 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:01.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:01.204 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:01.201+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:01.240 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:01.240 INFO:teuthology.orchestra.run.smithi031.stdout:2035 write 0x75f791e thru 0x760493a (0xd01d bytes) 2024-09-13T09:08:01.268 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:01.261+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:01.310 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:01.310 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:01.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:01.372 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:01.369+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:01.469 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:01.465+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:01.543 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:01.544 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:01.559 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:01.601 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:01.597+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:01.669 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:01.669 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:01.671 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:01.665+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:01.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:01.771 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:01.765+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:01.798 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:01.798 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:01.804 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:01.804 INFO:teuthology.orchestra.run.smithi031.stdout:2037 read 0x87301d2 thru 0x873f743 (0xf572 bytes) 2024-09-13T09:08:01.804 INFO:teuthology.orchestra.run.smithi031.stdout:2038 read 0x543c791 thru 0x54416a0 (0x4f10 bytes) 2024-09-13T09:08:01.805 INFO:teuthology.orchestra.run.smithi031.stdout:2040 writesame 0x3810342 thru 0x381ffc7 data_size 0x1(0xfc86 bytes) 2024-09-13T09:08:01.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:01.853 INFO:teuthology.orchestra.run.smithi031.stdout:2041 write 0x7088002 thru 0x7097440 (0xf43f bytes) 2024-09-13T09:08:01.856 INFO:teuthology.orchestra.run.smithi031.stdout:2042 write 0x239a593 thru 0x23a51e2 (0xac50 bytes) 2024-09-13T09:08:01.872 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:01.868+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:01.873 INFO:teuthology.orchestra.run.smithi031.stdout:2043 write 0xc54f4ca thru 0xc559bc3 (0xa6fa bytes) 2024-09-13T09:08:01.880 INFO:teuthology.orchestra.run.smithi031.stdout:2046 punch from 0x950e6d5 to 0x95166df, (0x800a bytes) 2024-09-13T09:08:01.880 INFO:teuthology.orchestra.run.smithi031.stdout:2047 punch from 0x6cf5ce8 to 0x6d01584, (0xb89c bytes) 2024-09-13T09:08:01.880 INFO:teuthology.orchestra.run.smithi031.stdout:2048 writesame 0x4452f31 thru 0x4456379 data_size 0x1(0x3449 bytes) 2024-09-13T09:08:01.890 INFO:teuthology.orchestra.run.smithi031.stdout:2050 writesame 0x5749fc0 thru 0x574a9d6 data_size 0x3(0xa17 bytes) 2024-09-13T09:08:01.906 INFO:teuthology.orchestra.run.smithi031.stdout:2051 write 0xe9dc832 thru 0xe9ec434 (0xfc03 bytes) 2024-09-13T09:08:01.919 INFO:teuthology.orchestra.run.smithi031.stdout:2053 trunc from 0xe9ec435 to 0x6eb2c25 2024-09-13T09:08:01.931 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:08:01.937 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:01.937 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:01.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:08:01.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:01.957 INFO:teuthology.orchestra.run.smithi031.stdout:2054 write 0x1da1c5e thru 0x1da8aee (0x6e91 bytes) 2024-09-13T09:08:01.959 INFO:teuthology.orchestra.run.smithi031.stdout:2056 read 0x18f6937 thru 0x18fbee1 (0x55ab bytes) 2024-09-13T09:08:01.967 INFO:teuthology.orchestra.run.smithi031.stdout:2057 trunc from 0x6eb2c25 to 0x52db0f 2024-09-13T09:08:01.974 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:01.969+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:02.014 INFO:teuthology.orchestra.run.smithi031.stdout:2060 write 0xe7ce5d6 thru 0xe7ddc26 (0xf651 bytes) 2024-09-13T09:08:02.025 INFO:teuthology.orchestra.run.smithi031.stdout:2061 trunc from 0xe7ddc27 to 0x890cf14 2024-09-13T09:08:02.036 INFO:teuthology.orchestra.run.smithi031.stdout:2063 write 0x5d524d4 thru 0x5d5c353 (0x9e80 bytes) 2024-09-13T09:08:02.044 INFO:teuthology.orchestra.run.smithi031.stdout:2064 trunc from 0x890cf14 to 0xdd40987 2024-09-13T09:08:02.056 INFO:teuthology.orchestra.run.smithi031.stdout:2066 punch from 0xb84ae81 to 0xb84c160, (0x12df bytes) 2024-09-13T09:08:02.056 INFO:teuthology.orchestra.run.smithi031.stdout:2067 writesame 0x4782dc7 thru 0x478481c data_size 0x1(0x1a56 bytes) 2024-09-13T09:08:02.063 INFO:teuthology.orchestra.run.smithi031.stdout:2068 write 0x46795d5 thru 0x468009f (0x6acb bytes) 2024-09-13T09:08:02.067 INFO:teuthology.orchestra.run.smithi031.stdout:2071 writesame 0xc02b85d thru 0xc0376c8 data_size 0xe(0xbe6c bytes) 2024-09-13T09:08:02.074 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:02.075 INFO:teuthology.orchestra.run.smithi031.stdout:2073 writesame 0x69f398c thru 0x69f43cc data_size 0x19(0xa41 bytes) 2024-09-13T09:08:02.075 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:02.069+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:02.076 INFO:teuthology.orchestra.run.smithi031.stdout:2075 clone 19 order 21 su 8192 sc 3 2024-09-13T09:08:02.077 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:02.077 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:02.092 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:02.176 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:02.169+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:02.242 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:02.242 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:02.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:02.277 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:02.273+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:02.364 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:02.364 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:02.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:02.378 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:02.372+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:02.479 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:02.473+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:02.498 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:02.498 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:02.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:02.579 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:02.573+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:02.638 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:02.638 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:02.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:02.680 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:02.677+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:02.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:02.772 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:02.773 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:02.781 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:02.776+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:02.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:02.882 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:02.882 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:02.882 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:02.877+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:02.898 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:02.898 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:02.914 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:02.915 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:02.929 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:08:02.934 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:02.934 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:02.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:08:02.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:08:02.964 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:02.964 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:02.966 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:02.966 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:02.966 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:02.966 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:02.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:02.983 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:02.977+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:02.984 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:02.985 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:03.021 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:03.021 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:03.052 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone18 from 0xdd40987 (overlap 0x52db0f) to 0x0 2024-09-13T09:08:03.057 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:03.058 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:03.083 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:03.077+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:03.113 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:03.113 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:03.113 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:03.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:03.164 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-13T09:08:03.207 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:03.204+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:03.284 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:03.280+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:03.345 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.345 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.212217838999997, 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:08:03.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:08:03.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:08:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:08:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:08:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.383935659, 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:08:03.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:08:03.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56973 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4501504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0950305629999999, 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091397946999999993, 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56973 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247918+0000", 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247921+0000", 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247927+0000", 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2389999999999999e-06 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247932+0000", 2024-09-13T09:08:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-13T09:08:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339077+0000", 2024-09-13T09:08:03.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091144366000000004 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339203+0000", 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126919 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339317+0000", 2024-09-13T09:08:03.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113205 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56975 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4515840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0948367960000001, 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091417921999999999, 2024-09-13T09:08:03.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56975 2024-09-13T09:08:03.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247936+0000", 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999824 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248117+0000", 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180827 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248127+0000", 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248133+0000", 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.19e-06 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339347+0000", 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091213581000000002 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339419+0000", 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.203e-05 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339530+0000", 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111359 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56976 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3681280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:03.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0947933989999998, 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091677039000000002, 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56976 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:03.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248155+0000", 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248159+0000", 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248166+0000", 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999999e-06 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248172+0000", 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339558+0000", 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091385818999999993 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339705+0000", 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014693599999999999 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339833+0000", 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012808200000000001 2024-09-13T09:08:03.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56990 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3795968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0916528969999999, 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091546566999999995, 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56990 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251298+0000", 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.041e-06 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251303+0000", 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-13T09:08:03.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251319+0000", 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5852e-05 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342591+0000", 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091271605000000006 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342728+0000", 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013662799999999999 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342843+0000", 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115442 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.382 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57022 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4334592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0915702700000001, 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091691988000000002, 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57022 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251382+0000", 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5659999999999998e-06 2024-09-13T09:08:03.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251391+0000", 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0170000000000002e-06 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251397+0000", 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.908e-06 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342870+0000", 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091473410000000005 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342945+0000", 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4175e-05 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343071+0000", 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012638499999999999 2024-09-13T09:08:03.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57026 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5113856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0915473329999998, 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091880383999999996, 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57026 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:03.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251404+0000", 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251410+0000", 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251415+0000", 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2129999999999999e-06 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343098+0000", 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091683073000000004 2024-09-13T09:08:03.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343171+0000", 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3149e-05 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343282+0000", 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110899 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57027 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4348928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0915296899999998, 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092079082000000007, 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57027 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251419+0000", 2024-09-13T09:08:03.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251421+0000", 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251427+0000", 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4419999999999997e-06 2024-09-13T09:08:03.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251432+0000", 2024-09-13T09:08:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7620000000000003e-06 2024-09-13T09:08:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343310+0000", 2024-09-13T09:08:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091878572000000006 2024-09-13T09:08:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343383+0000", 2024-09-13T09:08:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2368999999999998e-05 2024-09-13T09:08:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343499+0000", 2024-09-13T09:08:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116142 2024-09-13T09:08:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57028 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5128192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0915123449999999, 2024-09-13T09:08:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092372671000000003, 2024-09-13T09:08:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57028 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251439+0000", 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1730000000000002e-06 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251445+0000", 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251450+0000", 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343526+0000", 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092076621999999997 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343605+0000", 2024-09-13T09:08:03.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8830000000000002e-05 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343810+0000", 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204544 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0914958210000001, 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:08:03.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:08:03.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0914160229999998, 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:08:03.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.091166952, 2024-09-13T09:08:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:08:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:08:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:08:03.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0908538110000001, 2024-09-13T09:08:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:08:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:03.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:08:03.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:08:03.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0904005790000002, 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:08:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:08:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0903760650000001, 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:08:03.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:08:03.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:03.436 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.090110836, 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:08:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:08:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:08:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:08:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:08:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:08:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:08:03.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:08:03.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:03.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0898229869999998, 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:08:03.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:08:03.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57037 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0893867859999999, 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092241045999999993, 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57037 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253152+0000", 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995894 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253567+0000", 2024-09-13T09:08:03.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000414886 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253576+0000", 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6750000000000008e-06 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253582+0000", 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345610+0000", 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092027943000000001 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345682+0000", 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2311999999999995e-05 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345804+0000", 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121332 2024-09-13T09:08:03.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0893616829999999, 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:03.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:08:03.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:08:03.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:03.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:03.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:08:03.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:08:03.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:03.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:03.458 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:03.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:03.460 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:03.453+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:03.561 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:03.557+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71460 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.565897+0000", 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027344195000000002, 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027367826000000001, 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.611 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71460 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565897+0000", 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565897+0000", 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565896+0000", 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565900+0000", 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9759999999999997e-06 2024-09-13T09:08:03.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565911+0000", 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1048e-05 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565922+0000", 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0818e-05 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71525 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567269+0000", 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025971992999999999, 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026047364999999999, 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71525 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.615 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567269+0000", 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567269+0000", 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567268+0000", 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567271+0000", 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4540000000000001e-06 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567276+0000", 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.446e-06 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567281+0000", 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7790000000000003e-06 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71580 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568165+0000", 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025075511000000002, 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025185270999999999, 2024-09-13T09:08:03.618 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71580 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568165+0000", 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568165+0000", 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568164+0000", 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568176+0000", 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1616e-05 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568201+0000", 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4481000000000002e-05 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568213+0000", 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.257e-05 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.621 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71801 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569272+0000", 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023968757, 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.024109242999999999, 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71801 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569272+0000", 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569272+0000", 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569271+0000", 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569276+0000", 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3359999999999997e-06 2024-09-13T09:08:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569299+0000", 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3887999999999999e-05 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569319+0000", 2024-09-13T09:08:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9137e-05 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71885 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570301+0000", 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.022940089, 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023110770999999999, 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71885 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570301+0000", 2024-09-13T09:08:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570301+0000", 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570300+0000", 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570304+0000", 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5779999999999999e-06 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570327+0000", 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3278999999999999e-05 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570342+0000", 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4908e-05 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71950 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576809+0000", 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016431687, 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016634102000000001, 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71950 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576809+0000", 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576809+0000", 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576809+0000", 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576810+0000", 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.72e-06 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576820+0000", 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6670000000000008e-06 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576824+0000", 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.749e-06 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71995 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584673+0000", 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0085674749999999997, 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0088013369999999994, 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.634 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71995 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584673+0000", 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584673+0000", 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584673+0000", 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584675+0000", 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6780000000000001e-06 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584680+0000", 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8429999999999998e-06 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584686+0000", 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1769999999999997e-06 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72035 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592607+0000", 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00063385700000000004, 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00089683000000000004, 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72035 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592607+0000", 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592607+0000", 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592607+0000", 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592608+0000", 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5859999999999999e-06 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592612+0000", 2024-09-13T09:08:03.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1110000000000001e-06 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592616+0000", 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6059999999999999e-06 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71462 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.641 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.565952+0000", 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027289032000000001, 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027581600000000001, 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71462 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565952+0000", 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565952+0000", 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565951+0000", 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565954+0000", 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9859999999999999e-06 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565963+0000", 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3380000000000005e-06 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565972+0000", 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9539999999999993e-06 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.645 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71527 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567285+0000", 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025956013999999999, 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026279275000000001, 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71527 2024-09-13T09:08:03.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567285+0000", 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567285+0000", 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567284+0000", 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567287+0000", 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2479999999999999e-06 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567294+0000", 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2599999999999999e-06 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567298+0000", 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71582 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568224+0000", 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025016773999999999, 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025370936, 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71582 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568224+0000", 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568224+0000", 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568223+0000", 2024-09-13T09:08:03.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568228+0000", 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4560000000000002e-06 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568241+0000", 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3585e-05 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568262+0000", 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0905999999999999e-05 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71803 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569338+0000", 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023902986000000001, 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.024287953000000001, 2024-09-13T09:08:03.653 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71803 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569338+0000", 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569338+0000", 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569337+0000", 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569342+0000", 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5460000000000002e-06 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569355+0000", 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3518e-05 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569368+0000", 2024-09-13T09:08:03.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2745000000000001e-05 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71889 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570361+0000", 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.022879348000000001, 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023295720999999998, 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.657 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71889 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570361+0000", 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570361+0000", 2024-09-13T09:08:03.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570361+0000", 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570365+0000", 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8119999999999998e-06 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570379+0000", 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3823e-05 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570390+0000", 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0961999999999999e-05 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71951 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576827+0000", 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016413483999999999, 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016865524999999999, 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71951 2024-09-13T09:08:03.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576827+0000", 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576827+0000", 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576827+0000", 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576838+0000", 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1477999999999999e-05 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576850+0000", 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1433999999999999e-05 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576854+0000", 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3229999999999999e-06 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71997 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584691+0000", 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0085501910000000004, 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00903231, 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.665 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71997 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584691+0000", 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584691+0000", 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584690+0000", 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584692+0000", 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8220000000000001e-06 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584696+0000", 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2150000000000003e-06 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584699+0000", 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72036 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592619+0000", 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00062207300000000001, 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011333199999999999, 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72036 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.669 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592619+0000", 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592619+0000", 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592618+0000", 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592620+0000", 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6360000000000001e-06 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592624+0000", 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0709999999999996e-06 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592627+0000", 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3529999999999999e-06 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71464 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.565980+0000", 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027260851999999999, 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027801625, 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.673 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71464 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565980+0000", 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565980+0000", 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565979+0000", 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565983+0000", 2024-09-13T09:08:03.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3220000000000003e-06 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.565994+0000", 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0983e-05 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566005+0000", 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.079e-05 2024-09-13T09:08:03.676 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71529 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567302+0000", 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025938536000000002, 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.02650897, 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.677 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71529 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567302+0000", 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567302+0000", 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567302+0000", 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567304+0000", 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.57e-06 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567310+0000", 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4319999999999998e-06 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567314+0000", 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5739999999999997e-06 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71584 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568271+0000", 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024970147000000002, 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025577623000000001, 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.681 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71584 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568271+0000", 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568271+0000", 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568270+0000", 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568275+0000", 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7720000000000002e-06 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568288+0000", 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3454e-05 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568308+0000", 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9973e-05 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71806 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569378+0000", 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023862254999999999, 2024-09-13T09:08:03.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.02450072, 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71806 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569378+0000", 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569378+0000", 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569378+0000", 2024-09-13T09:08:03.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569382+0000", 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4619999999999998e-06 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569396+0000", 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3444e-05 2024-09-13T09:08:03.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569417+0000", 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0839000000000002e-05 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71891 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.689 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572451+0000", 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020789935999999998, 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021458067000000001, 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71891 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572451+0000", 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572451+0000", 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570398+0000", 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9979467 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572457+0000", 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002059773 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572470+0000", 2024-09-13T09:08:03.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2880999999999999e-05 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572478+0000", 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3379999999999999e-06 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71952 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.693 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576871+0000", 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016370118999999999, 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017067635000000001, 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71952 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576871+0000", 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576871+0000", 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576870+0000", 2024-09-13T09:08:03.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576872+0000", 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4080000000000001e-06 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576877+0000", 2024-09-13T09:08:03.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7450000000000003e-06 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576881+0000", 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7060000000000002e-06 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.697 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71999 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584710+0000", 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0085306770000000004, 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0092576599999999992, 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71999 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.698 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584710+0000", 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584710+0000", 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584710+0000", 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584712+0000", 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2900000000000001e-06 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584717+0000", 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.673e-06 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584720+0000", 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5360000000000001e-06 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72037 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592630+0000", 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00061055199999999999, 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0013665839999999999, 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72037 2024-09-13T09:08:03.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592630+0000", 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592630+0000", 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592630+0000", 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592632+0000", 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8109999999999999e-06 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592636+0000", 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1729999999999997e-06 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592639+0000", 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4070000000000002e-06 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.705 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71466 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566013+0000", 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027227811000000001, 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028014792, 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.706 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71466 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566013+0000", 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566013+0000", 2024-09-13T09:08:03.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566012+0000", 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566034+0000", 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1772000000000001e-05 2024-09-13T09:08:03.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566049+0000", 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5413e-05 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566068+0000", 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8267999999999999e-05 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.709 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71531 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567319+0000", 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025922181999999998, 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026738751000000002, 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.710 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71531 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567319+0000", 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567319+0000", 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567318+0000", 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567320+0000", 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2239999999999998e-06 2024-09-13T09:08:03.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567326+0000", 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3979999999999998e-06 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567331+0000", 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7679999999999999e-06 2024-09-13T09:08:03.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71586 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568319+0000", 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024922138, 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025775441, 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71586 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568319+0000", 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568319+0000", 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568318+0000", 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568322+0000", 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3340000000000003e-06 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568337+0000", 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5182e-05 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568358+0000", 2024-09-13T09:08:03.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0635000000000001e-05 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71808 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569427+0000", 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023813402000000001, 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.024696979000000001, 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.718 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71808 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569427+0000", 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569427+0000", 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569427+0000", 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569431+0000", 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3259999999999997e-06 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569443+0000", 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1744e-05 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569468+0000", 2024-09-13T09:08:03.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5632000000000001e-05 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71894 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572484+0000", 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020756885999999999, 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021674959000000001, 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.722 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71894 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572484+0000", 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572484+0000", 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572483+0000", 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572486+0000", 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2299999999999998e-06 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572497+0000", 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1440999999999999e-05 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572507+0000", 2024-09-13T09:08:03.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6390000000000004e-06 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71953 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580409+0000", 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012832174999999999, 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.013781326, 2024-09-13T09:08:03.726 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71953 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580409+0000", 2024-09-13T09:08:03.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580409+0000", 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576884+0000", 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9964752 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580413+0000", 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035293270000000001 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580423+0000", 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9629999999999993e-06 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580429+0000", 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8409999999999996e-06 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72001 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584723+0000", 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0085173450000000008, 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0094975040000000004, 2024-09-13T09:08:03.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72001 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584723+0000", 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584723+0000", 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584723+0000", 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584725+0000", 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.703e-06 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584729+0000", 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0860000000000004e-06 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584732+0000", 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5640000000000001e-06 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72038 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592642+0000", 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00059841500000000004, 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016128799999999999, 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.734 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72038 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592642+0000", 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592642+0000", 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592642+0000", 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592644+0000", 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6640000000000001e-06 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592648+0000", 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0840000000000002e-06 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592651+0000", 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3960000000000002e-06 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71468 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566074+0000", 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027166783999999999, 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028210789, 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.738 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71468 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566074+0000", 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566074+0000", 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566073+0000", 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566076+0000", 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.05e-06 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566083+0000", 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3820000000000001e-06 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566091+0000", 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3639999999999999e-06 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71533 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567334+0000", 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025906378000000001, 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026981799000000001, 2024-09-13T09:08:03.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71533 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567334+0000", 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567334+0000", 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567334+0000", 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567336+0000", 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567350+0000", 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3708e-05 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567355+0000", 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7419999999999997e-06 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71588 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568377+0000", 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024863433000000001, 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025968379999999999, 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.746 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71588 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568377+0000", 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568377+0000", 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568377+0000", 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:03.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568381+0000", 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6840000000000004e-06 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568421+0000", 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9721000000000001e-05 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568432+0000", 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0684e-05 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71811 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569480+0000", 2024-09-13T09:08:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023760999000000001, 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.024931594000000001, 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71811 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569480+0000", 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569480+0000", 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569479+0000", 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569503+0000", 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3601999999999999e-05 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569519+0000", 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6762e-05 2024-09-13T09:08:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569533+0000", 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4081e-05 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71897 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572511+0000", 2024-09-13T09:08:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020729971, 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021939597000000002, 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71897 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572511+0000", 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572511+0000", 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572510+0000", 2024-09-13T09:08:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572512+0000", 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0829999999999998e-06 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572517+0000", 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7060000000000003e-06 2024-09-13T09:08:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572521+0000", 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.822e-06 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71954 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580446+0000", 2024-09-13T09:08:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012795084, 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014036710000000001, 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71954 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580446+0000", 2024-09-13T09:08:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580446+0000", 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580445+0000", 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580448+0000", 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3700000000000002e-06 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580453+0000", 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.118e-06 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580456+0000", 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7519999999999998e-06 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72002 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584736+0000", 2024-09-13T09:08:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0085042720000000002, 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0097763360000000001, 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72002 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584736+0000", 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584736+0000", 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584736+0000", 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584738+0000", 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.764e-06 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584742+0000", 2024-09-13T09:08:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0929999999999996e-06 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584745+0000", 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3629999999999998e-06 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72039 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592654+0000", 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00058665900000000001, 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001889251, 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72039 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592654+0000", 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592654+0000", 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592654+0000", 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592655+0000", 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6029999999999999e-06 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592659+0000", 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1019999999999999e-06 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592664+0000", 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2350000000000001e-06 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.770 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71470 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566099+0000", 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027142076000000001, 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028474846000000002, 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71470 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.771 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566099+0000", 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566099+0000", 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566098+0000", 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566102+0000", 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3529999999999997e-06 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566113+0000", 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0922e-05 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566122+0000", 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7940000000000008e-06 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71535 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567359+0000", 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025881814999999999, 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027248193, 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71535 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567359+0000", 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567359+0000", 2024-09-13T09:08:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567358+0000", 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567361+0000", 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.272e-06 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567366+0000", 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.536e-06 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567371+0000", 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6920000000000001e-06 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71590 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568438+0000", 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024803056, 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026198809999999999, 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71590 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568438+0000", 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568438+0000", 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568437+0000", 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568440+0000", 2024-09-13T09:08:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3919999999999999e-06 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568447+0000", 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2099999999999998e-06 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568451+0000", 2024-09-13T09:08:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6419999999999998e-06 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71814 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569550+0000", 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023690349999999999, 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025116095000000001, 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71814 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569550+0000", 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569550+0000", 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569550+0000", 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569553+0000", 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4999999999999999e-06 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569560+0000", 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6159999999999998e-06 2024-09-13T09:08:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569565+0000", 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.028e-06 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71899 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572524+0000", 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020716760000000001, 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022174223999999999, 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71899 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572524+0000", 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572524+0000", 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572524+0000", 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572525+0000", 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.697e-06 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572530+0000", 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.313e-06 2024-09-13T09:08:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572533+0000", 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.625e-06 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71955 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580460+0000", 2024-09-13T09:08:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012781176999999999, 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014271768000000001, 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71955 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580460+0000", 2024-09-13T09:08:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580460+0000", 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580459+0000", 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580461+0000", 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0159999999999998e-06 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580467+0000", 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580471+0000", 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4580000000000002e-06 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72003 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584748+0000", 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0084924519999999993, 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010013764, 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72003 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584748+0000", 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584748+0000", 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584748+0000", 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584750+0000", 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.671e-06 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584754+0000", 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8239999999999999e-06 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584758+0000", 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3230000000000002e-06 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72040 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592667+0000", 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00057408700000000003, 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021260329999999998, 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72040 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592667+0000", 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592667+0000", 2024-09-13T09:08:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592666+0000", 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592668+0000", 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.672e-06 2024-09-13T09:08:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592672+0000", 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0820000000000001e-06 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592676+0000", 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71472 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566132+0000", 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027108910999999999, 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028698125000000001, 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71472 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566132+0000", 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566132+0000", 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566131+0000", 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566135+0000", 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6959999999999998e-06 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566149+0000", 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3788000000000001e-05 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566157+0000", 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2430000000000005e-06 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71536 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567375+0000", 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025865901, 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027484877000000001, 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71536 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567375+0000", 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567375+0000", 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567374+0000", 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567377+0000", 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.216e-06 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567382+0000", 2024-09-13T09:08:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2000000000000002e-06 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567386+0000", 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4259999999999996e-06 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71592 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568455+0000", 2024-09-13T09:08:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024785418, 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026436370000000001, 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71592 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568455+0000", 2024-09-13T09:08:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568455+0000", 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568455+0000", 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568457+0000", 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568463+0000", 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.417e-06 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568467+0000", 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4279999999999998e-06 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71817 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569569+0000", 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023671557999999999, 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025351911000000001, 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71817 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569569+0000", 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569569+0000", 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569569+0000", 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569571+0000", 2024-09-13T09:08:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5500000000000001e-06 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569576+0000", 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2100000000000001e-06 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569581+0000", 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5560000000000001e-06 2024-09-13T09:08:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71901 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572536+0000", 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020704528999999999, 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022417408999999999, 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71901 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572536+0000", 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572536+0000", 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572536+0000", 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572538+0000", 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.84e-06 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572544+0000", 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7930000000000003e-06 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572547+0000", 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4489999999999999e-06 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71956 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580474+0000", 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012767120999999999, 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014511412, 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71956 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580474+0000", 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580474+0000", 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580473+0000", 2024-09-13T09:08:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580475+0000", 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.809e-06 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580479+0000", 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1880000000000004e-06 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580483+0000", 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3330000000000001e-06 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72004 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584761+0000", 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0084800550000000002, 2024-09-13T09:08:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010254835, 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72004 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584761+0000", 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584761+0000", 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584760+0000", 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584771+0000", 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0913999999999999e-05 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584776+0000", 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9250000000000003e-06 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584780+0000", 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6399999999999999e-06 2024-09-13T09:08:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72041 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592678+0000", 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00056231100000000002, 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023661870000000001, 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72041 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592678+0000", 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592678+0000", 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592678+0000", 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592680+0000", 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.719e-06 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592684+0000", 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0999999999999997e-06 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592688+0000", 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0679999999999998e-06 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71474 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566197+0000", 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027044102, 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028877127999999998, 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71474 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566197+0000", 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566197+0000", 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566196+0000", 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566199+0000", 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.118e-06 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566207+0000", 2024-09-13T09:08:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7500000000000003e-06 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566216+0000", 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2390000000000008e-06 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71537 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567390+0000", 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025850583, 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027714868, 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71537 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567390+0000", 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567390+0000", 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567390+0000", 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567392+0000", 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1449999999999998e-06 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567397+0000", 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2279999999999998e-06 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567402+0000", 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4220000000000002e-06 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71594 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568471+0000", 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024769646999999999, 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026663438000000001, 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71594 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568471+0000", 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568471+0000", 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568471+0000", 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568473+0000", 2024-09-13T09:08:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.356e-06 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568478+0000", 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3399999999999997e-06 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568483+0000", 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3560000000000003e-06 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71820 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569585+0000", 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023655908999999999, 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025582766999999999, 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71820 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569585+0000", 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569585+0000", 2024-09-13T09:08:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569584+0000", 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569587+0000", 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2220000000000001e-06 2024-09-13T09:08:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569592+0000", 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3600000000000004e-06 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569596+0000", 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.386e-06 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71904 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572550+0000", 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020690627999999999, 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022652595000000001, 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71904 2024-09-13T09:08:03.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572550+0000", 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572550+0000", 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572550+0000", 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572551+0000", 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7209999999999999e-06 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572556+0000", 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3100000000000002e-06 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572559+0000", 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.444e-06 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71957 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580487+0000", 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012754188, 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014747509000000001, 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71957 2024-09-13T09:08:03.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580487+0000", 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580487+0000", 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580486+0000", 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580488+0000", 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.767e-06 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580493+0000", 2024-09-13T09:08:03.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8550000000000001e-06 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580499+0000", 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.855 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72006 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584783+0000", 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0084576410000000001, 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010479885, 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72006 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.856 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584783+0000", 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584783+0000", 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584783+0000", 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584785+0000", 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.81e-06 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584789+0000", 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0400000000000003e-06 2024-09-13T09:08:03.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584792+0000", 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3670000000000001e-06 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72042 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592691+0000", 2024-09-13T09:08:03.859 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00054980299999999997, 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026024720000000002, 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72042 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592691+0000", 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592691+0000", 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592691+0000", 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592692+0000", 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.64e-06 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592696+0000", 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.109e-06 2024-09-13T09:08:03.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592700+0000", 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1860000000000001e-06 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71476 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566226+0000", 2024-09-13T09:08:03.863 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.027014781000000002, 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029097074000000001, 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71476 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566226+0000", 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566226+0000", 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566225+0000", 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566229+0000", 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566240+0000", 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1114000000000001e-05 2024-09-13T09:08:03.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566251+0000", 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0812000000000001e-05 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71538 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.867 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567405+0000", 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025835396, 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027950572999999999, 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71538 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.868 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567405+0000", 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567405+0000", 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567405+0000", 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567421+0000", 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5794000000000002e-05 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567429+0000", 2024-09-13T09:08:03.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4540000000000007e-06 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567434+0000", 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9239999999999998e-06 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71596 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.871 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568487+0000", 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024754001000000001, 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026898741, 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71596 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568487+0000", 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568487+0000", 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568486+0000", 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568488+0000", 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0700000000000001e-06 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568494+0000", 2024-09-13T09:08:03.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2340000000000002e-06 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568498+0000", 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3059999999999999e-06 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.875 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71824 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569600+0000", 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023640432999999999, 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025816302999999999, 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71824 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569600+0000", 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569600+0000", 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569600+0000", 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569602+0000", 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.21e-06 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569607+0000", 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2440000000000001e-06 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569612+0000", 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.42e-06 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71905 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572562+0000", 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020678338000000001, 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022883384999999999, 2024-09-13T09:08:03.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71905 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572562+0000", 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572562+0000", 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572562+0000", 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572564+0000", 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8649999999999999e-06 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572568+0000", 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1989999999999999e-06 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572574+0000", 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1879999999999999e-06 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.882 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71958 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580502+0000", 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012738728, 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014972864000000001, 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71958 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580502+0000", 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580502+0000", 2024-09-13T09:08:03.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580502+0000", 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580503+0000", 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7179999999999999e-06 2024-09-13T09:08:03.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580510+0000", 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3300000000000004e-06 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580513+0000", 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4209999999999999e-06 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.886 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72008 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588492+0000", 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0047489009999999998, 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0070917710000000002, 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72008 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588492+0000", 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588492+0000", 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584794+0000", 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9963026 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588496+0000", 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037012949999999998 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588506+0000", 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0202e-05 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588512+0000", 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9329999999999998e-06 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.890 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72043 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592730+0000", 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00051071299999999999, 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028835140000000002, 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72043 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592730+0000", 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592730+0000", 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592729+0000", 2024-09-13T09:08:03.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592732+0000", 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5770000000000001e-06 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592738+0000", 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7590000000000003e-06 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592742+0000", 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9090000000000002e-06 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71478 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566259+0000", 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026982104999999999, 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029387092, 2024-09-13T09:08:03.894 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71478 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566259+0000", 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566259+0000", 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566258+0000", 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566262+0000", 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8700000000000002e-06 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566273+0000", 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0975000000000001e-05 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566282+0000", 2024-09-13T09:08:03.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.414e-06 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71540 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567458+0000", 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025782680999999998, 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028218713999999999, 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.898 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71540 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567458+0000", 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567458+0000", 2024-09-13T09:08:03.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567457+0000", 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567461+0000", 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.532e-06 2024-09-13T09:08:03.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567474+0000", 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2208000000000001e-05 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567485+0000", 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1209999999999999e-05 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.901 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71598 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568502+0000", 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024739059000000001, 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027206668, 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.902 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71598 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568502+0000", 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568502+0000", 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568501+0000", 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568503+0000", 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:08:03.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568511+0000", 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5639999999999999e-06 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568515+0000", 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5260000000000004e-06 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.905 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71826 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569615+0000", 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023625246999999999, 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026125103, 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71826 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569615+0000", 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569615+0000", 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569615+0000", 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569617+0000", 2024-09-13T09:08:03.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1560000000000002e-06 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569623+0000", 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2800000000000003e-06 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569627+0000", 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.228e-06 2024-09-13T09:08:03.909 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71907 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572578+0000", 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020663194999999999, 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023194105999999999, 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.910 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71907 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572578+0000", 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572578+0000", 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572577+0000", 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572584+0000", 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9639999999999997e-06 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572589+0000", 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.454e-06 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572593+0000", 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3989999999999997e-06 2024-09-13T09:08:03.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71959 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580516+0000", 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012724521000000001, 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.015284320000000001, 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71959 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580516+0000", 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580516+0000", 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580516+0000", 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580518+0000", 2024-09-13T09:08:03.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.666e-06 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580522+0000", 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1649999999999999e-06 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580525+0000", 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3129999999999999e-06 2024-09-13T09:08:03.917 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72010 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588531+0000", 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0047096020000000002, 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.007299837, 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.918 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72010 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588531+0000", 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588531+0000", 2024-09-13T09:08:03.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588531+0000", 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588533+0000", 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.17e-06 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588538+0000", 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8670000000000002e-06 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588542+0000", 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9099999999999998e-06 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.921 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72044 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592745+0000", 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.000495682, 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003114669, 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72044 2024-09-13T09:08:03.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592745+0000", 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592745+0000", 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592745+0000", 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592746+0000", 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.877e-06 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592752+0000", 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6760000000000004e-06 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592756+0000", 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4989999999999999e-06 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71481 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566317+0000", 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026924136000000001, 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029576270000000002, 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71481 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.926 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566317+0000", 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566317+0000", 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566316+0000", 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566319+0000", 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566327+0000", 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6750000000000002e-06 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566337+0000", 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.91e-06 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71541 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567494+0000", 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025746992999999999, 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028434812, 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.930 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71541 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567494+0000", 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567494+0000", 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567493+0000", 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567497+0000", 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2760000000000002e-06 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567514+0000", 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6178999999999999e-05 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567523+0000", 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5599999999999999e-06 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71600 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568519+0000", 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024721346000000002, 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027440506999999999, 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71600 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568519+0000", 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568519+0000", 2024-09-13T09:08:03.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568519+0000", 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568521+0000", 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1919999999999999e-06 2024-09-13T09:08:03.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568526+0000", 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568531+0000", 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3719999999999998e-06 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.937 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71828 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569630+0000", 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023610261, 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026361160000000002, 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71828 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569630+0000", 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569630+0000", 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569630+0000", 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569632+0000", 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3020000000000002e-06 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569639+0000", 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3899999999999998e-06 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569643+0000", 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4190000000000004e-06 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71910 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572596+0000", 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020644520999999999, 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023424681999999999, 2024-09-13T09:08:03.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71910 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572596+0000", 2024-09-13T09:08:03.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572596+0000", 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572596+0000", 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572598+0000", 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6980000000000001e-06 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572602+0000", 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3229999999999999e-06 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572605+0000", 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.343e-06 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71960 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580528+0000", 2024-09-13T09:08:03.946 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012712807, 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.015524059, 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71960 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580528+0000", 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580528+0000", 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580528+0000", 2024-09-13T09:08:03.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580529+0000", 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7570000000000001e-06 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580539+0000", 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9769999999999995e-06 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580543+0000", 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.602e-06 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72012 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588545+0000", 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0046959050000000002, 2024-09-13T09:08:03.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0075378850000000002, 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72012 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588545+0000", 2024-09-13T09:08:03.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588545+0000", 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588544+0000", 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588546+0000", 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8700000000000001e-06 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588550+0000", 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1999999999999996e-06 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588554+0000", 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4470000000000002e-06 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72045 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592759+0000", 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00048218300000000001, 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00335513, 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72045 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592759+0000", 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592759+0000", 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592758+0000", 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592764+0000", 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.541e-06 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592768+0000", 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4070000000000003e-06 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592772+0000", 2024-09-13T09:08:03.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5410000000000001e-06 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71483 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566345+0000", 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026895571, 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029797832, 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.958 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71483 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566345+0000", 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566345+0000", 2024-09-13T09:08:03.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566344+0000", 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566348+0000", 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0659999999999997e-06 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566360+0000", 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566371+0000", 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0916e-05 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71542 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567533+0000", 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025707856000000001, 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028641264999999999, 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71542 2024-09-13T09:08:03.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567533+0000", 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567533+0000", 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567532+0000", 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567536+0000", 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0119999999999999e-06 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567549+0000", 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3232e-05 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567558+0000", 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3219999999999993e-06 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.965 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71602 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568535+0000", 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024706148000000001, 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027669022000000001, 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71602 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568535+0000", 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568535+0000", 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568534+0000", 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568536+0000", 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2000000000000001e-06 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568542+0000", 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2379999999999997e-06 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568546+0000", 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.42e-06 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71831 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569647+0000", 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023593659999999999, 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026591507, 2024-09-13T09:08:03.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71831 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569647+0000", 2024-09-13T09:08:03.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569647+0000", 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569647+0000", 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569649+0000", 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1040000000000001e-06 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569654+0000", 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4840000000000003e-06 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569659+0000", 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.386e-06 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71913 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.572608+0000", 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020632554000000001, 2024-09-13T09:08:03.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023661657999999999, 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71913 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572608+0000", 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572608+0000", 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572608+0000", 2024-09-13T09:08:03.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572610+0000", 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.697e-06 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572615+0000", 2024-09-13T09:08:03.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.04e-06 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572618+0000", 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.377e-06 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.978 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71961 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580546+0000", 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012694560000000001, 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.015759825000000002, 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71961 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580546+0000", 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580546+0000", 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580546+0000", 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580548+0000", 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.79e-06 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580554+0000", 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2829999999999998e-06 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580558+0000", 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0670000000000002e-06 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72014 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588557+0000", 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0046836459999999996, 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0077786139999999997, 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72014 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.983 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588557+0000", 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588557+0000", 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588557+0000", 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588558+0000", 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7129999999999999e-06 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588563+0000", 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1169999999999997e-06 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588566+0000", 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3850000000000002e-06 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72046 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592775+0000", 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00046614600000000002, 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035900369999999999, 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72046 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592775+0000", 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592775+0000", 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592774+0000", 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592776+0000", 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7859999999999999e-06 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592781+0000", 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8210000000000001e-06 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592784+0000", 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4300000000000002e-06 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.990 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71485 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566381+0000", 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026859669999999999, 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030033351999999999, 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71485 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.991 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566381+0000", 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566381+0000", 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566380+0000", 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566384+0000", 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0260000000000001e-06 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566395+0000", 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1022000000000001e-05 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566406+0000", 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0611999999999999e-05 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71543 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567568+0000", 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025673047000000001, 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028877219999999999, 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71543 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.995 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567568+0000", 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567568+0000", 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567567+0000", 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567571+0000", 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4519999999999999e-06 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:03.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567585+0000", 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3178e-05 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567595+0000", 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9760000000000007e-06 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71604 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:03.998 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568550+0000", 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024690859999999999, 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027927378999999999, 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71604 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:03.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568550+0000", 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568550+0000", 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568549+0000", 2024-09-13T09:08:04.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568558+0000", 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7280000000000001e-06 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568572+0000", 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3380000000000001e-05 2024-09-13T09:08:04.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568578+0000", 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0229999999999998e-06 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71833 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569663+0000", 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023578175999999999, 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026845507000000001, 2024-09-13T09:08:04.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71833 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569663+0000", 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569663+0000", 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569662+0000", 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569665+0000", 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4200000000000001e-06 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569670+0000", 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3460000000000001e-06 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569674+0000", 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.34e-06 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71916 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576467+0000", 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016774193999999999, 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.020072684, 2024-09-13T09:08:04.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71916 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576467+0000", 2024-09-13T09:08:04.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576467+0000", 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.572621+0000", 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9961543 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576471+0000", 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003850268 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576482+0000", 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1389000000000001e-05 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576488+0000", 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8109999999999999e-06 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71962 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580561+0000", 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012679776, 2024-09-13T09:08:04.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016007598000000001, 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71962 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580561+0000", 2024-09-13T09:08:04.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580561+0000", 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580561+0000", 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580563+0000", 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9539999999999998e-06 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580567+0000", 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2560000000000004e-06 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580570+0000", 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2820000000000001e-06 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72015 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588570+0000", 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00467023, 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0080286550000000009, 2024-09-13T09:08:04.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72015 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588570+0000", 2024-09-13T09:08:04.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588570+0000", 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588570+0000", 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588572+0000", 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1169999999999998e-06 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588576+0000", 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.216e-06 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588580+0000", 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72047 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592787+0000", 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00045351400000000002, 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038412279999999999, 2024-09-13T09:08:04.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72047 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592787+0000", 2024-09-13T09:08:04.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592787+0000", 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592787+0000", 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592789+0000", 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.736e-06 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592793+0000", 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1520000000000002e-06 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592796+0000", 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4089999999999999e-06 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71487 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566437+0000", 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026803711000000001, 2024-09-13T09:08:04.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030222206000000001, 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71487 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566437+0000", 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566437+0000", 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566437+0000", 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566440+0000", 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.016e-06 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566447+0000", 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6340000000000001e-06 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566456+0000", 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71544 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567602+0000", 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025638819, 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029086529999999999, 2024-09-13T09:08:04.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71544 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567602+0000", 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567602+0000", 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567601+0000", 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567605+0000", 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0319999999999997e-06 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567631+0000", 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5687e-05 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567640+0000", 2024-09-13T09:08:04.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.986e-06 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71606 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568610+0000", 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024630683, 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028109064999999999, 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.030 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71606 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568610+0000", 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568610+0000", 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568609+0000", 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568614+0000", 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9679999999999997e-06 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568639+0000", 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4405000000000001e-05 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568652+0000", 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2976000000000001e-05 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71835 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569679+0000", 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023561746000000001, 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027069682000000001, 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.034 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71835 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569679+0000", 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569679+0000", 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569679+0000", 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569681+0000", 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1040000000000001e-06 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569686+0000", 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3700000000000003e-06 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569690+0000", 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3039999999999998e-06 2024-09-13T09:08:04.037 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71925 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576505+0000", 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016735225999999999, 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.020276082000000001, 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.038 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71925 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576505+0000", 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576505+0000", 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576505+0000", 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576509+0000", 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9249999999999997e-06 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576517+0000", 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6950000000000006e-06 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576524+0000", 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7190000000000004e-06 2024-09-13T09:08:04.041 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71963 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580574+0000", 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012666798999999999, 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016269510000000001, 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.042 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71963 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580574+0000", 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580574+0000", 2024-09-13T09:08:04.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580574+0000", 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580580+0000", 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.127e-06 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580590+0000", 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8369999999999992e-06 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580593+0000", 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6559999999999998e-06 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72016 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588584+0000", 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0046563749999999999, 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0083005449999999995, 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72016 2024-09-13T09:08:04.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588584+0000", 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588584+0000", 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588584+0000", 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588586+0000", 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.801e-06 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588590+0000", 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588593+0000", 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3119999999999998e-06 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.049 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72048 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592799+0000", 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00044166300000000002, 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.004117725, 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72048 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592799+0000", 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592799+0000", 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592799+0000", 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592800+0000", 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5290000000000001e-06 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592804+0000", 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0160000000000002e-06 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592808+0000", 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.399e-06 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71489 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566466+0000", 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026774936999999999, 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.03048232, 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71489 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.054 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566466+0000", 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566466+0000", 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566465+0000", 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566469+0000", 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9720000000000003e-06 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566480+0000", 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.11e-05 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566491+0000", 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0718e-05 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71545 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567658+0000", 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025582296000000001, 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029321066999999999, 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.058 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71545 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567658+0000", 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567658+0000", 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567658+0000", 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567662+0000", 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.318e-06 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567685+0000", 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2872999999999999e-05 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567694+0000", 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1139999999999995e-06 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.061 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71608 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568661+0000", 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024579956, 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028347998999999999, 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.062 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71608 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568661+0000", 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568661+0000", 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568660+0000", 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568664+0000", 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3680000000000004e-06 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568678+0000", 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3108000000000001e-05 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568688+0000", 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0889e-05 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71838 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569694+0000", 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023546439999999998, 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027347865999999998, 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71838 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.066 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569694+0000", 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569694+0000", 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569694+0000", 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569697+0000", 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5370000000000002e-06 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569708+0000", 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0183999999999999e-05 2024-09-13T09:08:04.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569718+0000", 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0767999999999999e-05 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71929 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576530+0000", 2024-09-13T09:08:04.069 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016710809, 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.020541177000000001, 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71929 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576530+0000", 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576530+0000", 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576530+0000", 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576532+0000", 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9999999999999999e-06 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576536+0000", 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8060000000000002e-06 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576540+0000", 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5700000000000001e-06 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71964 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580597+0000", 2024-09-13T09:08:04.073 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012644163, 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016505186000000002, 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71964 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580597+0000", 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580597+0000", 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580596+0000", 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580598+0000", 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7099999999999999e-06 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580603+0000", 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0209999999999999e-06 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580606+0000", 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2820000000000001e-06 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72017 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588596+0000", 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0046444829999999996, 2024-09-13T09:08:04.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0085345950000000007, 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72017 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588596+0000", 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588596+0000", 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588596+0000", 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588598+0000", 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.657e-06 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588602+0000", 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0620000000000002e-06 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588605+0000", 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4319999999999999e-06 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.080 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72049 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592810+0000", 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00043021299999999998, 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.004350764, 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72049 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592810+0000", 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592810+0000", 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592810+0000", 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592812+0000", 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.556e-06 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592816+0000", 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0470000000000004e-06 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592819+0000", 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2849999999999999e-06 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71491 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566502+0000", 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0267388, 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.03068829, 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71491 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566502+0000", 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566502+0000", 2024-09-13T09:08:04.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566501+0000", 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566513+0000", 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1238e-05 2024-09-13T09:08:04.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566524+0000", 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566535+0000", 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0842000000000001e-05 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.088 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71546 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567704+0000", 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025536261000000001, 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029516234999999998, 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71546 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567704+0000", 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567704+0000", 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567704+0000", 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567708+0000", 2024-09-13T09:08:04.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.126e-06 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567725+0000", 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7104000000000001e-05 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567734+0000", 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1570000000000006e-06 2024-09-13T09:08:04.092 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71610 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568696+0000", 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024544335, 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028561705999999999, 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71610 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568696+0000", 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568696+0000", 2024-09-13T09:08:04.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568696+0000", 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568700+0000", 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2980000000000002e-06 2024-09-13T09:08:04.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568714+0000", 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3570000000000001e-05 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568724+0000", 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0516000000000001e-05 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71840 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569741+0000", 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023499282999999999, 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.027546434000000002, 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71840 2024-09-13T09:08:04.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569741+0000", 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569741+0000", 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569741+0000", 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569745+0000", 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3540000000000002e-06 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569759+0000", 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3861e-05 2024-09-13T09:08:04.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569768+0000", 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0580000000000004e-06 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71932 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576543+0000", 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016697558000000001, 2024-09-13T09:08:04.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.020776540999999999, 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71932 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576543+0000", 2024-09-13T09:08:04.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576543+0000", 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576543+0000", 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576545+0000", 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7e-06 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576549+0000", 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2579999999999997e-06 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576552+0000", 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.32e-06 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.103 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71965 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580609+0000", 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012631417000000001, 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016740685000000002, 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.104 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71965 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580609+0000", 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580609+0000", 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580609+0000", 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580611+0000", 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.728e-06 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580616+0000", 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5350000000000004e-06 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580620+0000", 2024-09-13T09:08:04.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5949999999999999e-06 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72018 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588608+0000", 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0046328569999999998, 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0087726000000000002, 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72018 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588608+0000", 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588608+0000", 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588608+0000", 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588609+0000", 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.663e-06 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588613+0000", 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2150000000000003e-06 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588617+0000", 2024-09-13T09:08:04.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3950000000000001e-06 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72050 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596446+0000", 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9967942, 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00096593799999999995, 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.112 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72050 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596446+0000", 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596446+0000", 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592822+0000", 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9963756 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596451+0000", 2024-09-13T09:08:04.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036289289999999999 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596460+0000", 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596465+0000", 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.115 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71493 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566569+0000", 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026671501, 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.03087436, 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.116 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71493 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566569+0000", 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566569+0000", 2024-09-13T09:08:04.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566569+0000", 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566572+0000", 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.078e-06 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566580+0000", 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8120000000000008e-06 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566588+0000", 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.884e-06 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71548 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567743+0000", 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025497323999999998, 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029729997000000001, 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71548 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567743+0000", 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567743+0000", 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567743+0000", 2024-09-13T09:08:04.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567747+0000", 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1779999999999996e-06 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567760+0000", 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3416e-05 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567781+0000", 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0256000000000001e-05 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71612 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568731+0000", 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024509221000000001, 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028772764999999999, 2024-09-13T09:08:04.123 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71612 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568731+0000", 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568731+0000", 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568731+0000", 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568735+0000", 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9899999999999999e-06 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568746+0000", 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1241e-05 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568755+0000", 2024-09-13T09:08:04.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5820000000000002e-06 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71842 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569778+0000", 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023463037999999999, 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.02775617, 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71842 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569778+0000", 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569778+0000", 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569777+0000", 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569781+0000", 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.766e-06 2024-09-13T09:08:04.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569792+0000", 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1637999999999999e-05 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569803+0000", 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.081e-05 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71934 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576555+0000", 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016685678999999998, 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021009607999999999, 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71934 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.131 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576555+0000", 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576555+0000", 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576555+0000", 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576556+0000", 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7099999999999999e-06 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576561+0000", 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1729999999999997e-06 2024-09-13T09:08:04.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576564+0000", 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.444e-06 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71966 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580645+0000", 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012595688000000001, 2024-09-13T09:08:04.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016950579, 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71966 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580645+0000", 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580645+0000", 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580645+0000", 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580647+0000", 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9999999999999999e-06 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580651+0000", 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7500000000000003e-06 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580655+0000", 2024-09-13T09:08:04.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5240000000000001e-06 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72019 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588620+0000", 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0046210289999999996, 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0090104299999999998, 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.138 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72019 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588620+0000", 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588620+0000", 2024-09-13T09:08:04.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588619+0000", 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588621+0000", 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.623e-06 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588625+0000", 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1829999999999996e-06 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588628+0000", 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3129999999999999e-06 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72051 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596483+0000", 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9967575, 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011787379999999999, 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.142 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72051 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596483+0000", 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596483+0000", 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596483+0000", 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596485+0000", 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0480000000000001e-06 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.144 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596490+0000", 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.797e-06 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596493+0000", 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7720000000000001e-06 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.145 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71495 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566597+0000", 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026643956999999999, 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031094634999999999, 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71495 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566597+0000", 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566597+0000", 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566596+0000", 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566600+0000", 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1819999999999999e-06 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566611+0000", 2024-09-13T09:08:04.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1005000000000001e-05 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566620+0000", 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71550 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567789+0000", 2024-09-13T09:08:04.149 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025451633000000001, 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029934697, 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71550 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567789+0000", 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567789+0000", 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567788+0000", 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567793+0000", 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6160000000000004e-06 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567808+0000", 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5138e-05 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567818+0000", 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7559999999999994e-06 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71614 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568762+0000", 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024478798, 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028992270000000001, 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71614 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568762+0000", 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568762+0000", 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568761+0000", 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568765+0000", 2024-09-13T09:08:04.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.89e-06 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568791+0000", 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5471999999999999e-05 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568803+0000", 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2641e-05 2024-09-13T09:08:04.156 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71846 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569828+0000", 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023412233000000001, 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028072508999999999, 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.157 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71846 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569828+0000", 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569828+0000", 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569827+0000", 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569833+0000", 2024-09-13T09:08:04.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.626e-06 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569848+0000", 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4946000000000001e-05 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569861+0000", 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2704000000000001e-05 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.160 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71935 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576568+0000", 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016672643000000001, 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021363219999999999, 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71935 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576568+0000", 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576568+0000", 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576568+0000", 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576569+0000", 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7239999999999999e-06 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576574+0000", 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1949999999999996e-06 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576577+0000", 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3230000000000002e-06 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71967 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580658+0000", 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012582874000000001, 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017306973, 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71967 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.165 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580658+0000", 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580658+0000", 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580658+0000", 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580659+0000", 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7549999999999999e-06 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580663+0000", 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1949999999999996e-06 2024-09-13T09:08:04.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580668+0000", 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8559999999999998e-06 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72020 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588633+0000", 2024-09-13T09:08:04.168 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0046077089999999998, 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0093607629999999994, 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72020 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588633+0000", 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588633+0000", 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588633+0000", 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588634+0000", 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6759999999999999e-06 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588639+0000", 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1489999999999996e-06 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588642+0000", 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3380000000000001e-06 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72052 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596496+0000", 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9967442, 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001537917, 2024-09-13T09:08:04.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72052 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596496+0000", 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596496+0000", 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596496+0000", 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596498+0000", 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.691e-06 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596504+0000", 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6390000000000003e-06 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596508+0000", 2024-09-13T09:08:04.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3809999999999999e-06 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71497 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566630+0000", 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026611136000000001, 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031436758000000002, 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.176 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71497 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566630+0000", 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566630+0000", 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566629+0000", 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566633+0000", 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8500000000000004e-06 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566646+0000", 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3665999999999999e-05 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566659+0000", 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2754e-05 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.179 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71552 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567826+0000", 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025415126999999999, 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.03027206, 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.180 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71552 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567826+0000", 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567826+0000", 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567825+0000", 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567829+0000", 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.216e-06 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567843+0000", 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3763e-05 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567855+0000", 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.199e-05 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71616 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568812+0000", 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024428782999999999, 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029315991, 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71616 2024-09-13T09:08:04.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568812+0000", 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568812+0000", 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568811+0000", 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568815+0000", 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1960000000000001e-06 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568829+0000", 2024-09-13T09:08:04.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568850+0000", 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0530000000000002e-05 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71848 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.187 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569871+0000", 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023369402000000001, 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028287922, 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71848 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569871+0000", 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569871+0000", 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569871+0000", 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569874+0000", 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7299999999999999e-06 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569886+0000", 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1627999999999999e-05 2024-09-13T09:08:04.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569916+0000", 2024-09-13T09:08:04.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0260000000000002e-05 2024-09-13T09:08:04.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.191 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71936 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576580+0000", 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016660984, 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021610312, 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.194 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71936 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576580+0000", 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576580+0000", 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576579+0000", 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576581+0000", 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7460000000000001e-06 2024-09-13T09:08:04.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576585+0000", 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2050000000000004e-06 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576589+0000", 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3869999999999999e-06 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.197 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71968 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580671+0000", 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012569478, 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017547956999999999, 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.198 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71968 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580671+0000", 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580671+0000", 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580671+0000", 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580673+0000", 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7269999999999999e-06 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580677+0000", 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0609999999999997e-06 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580680+0000", 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4199999999999999e-06 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.201 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72021 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588645+0000", 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0045959110000000003, 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.009604998, 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72021 2024-09-13T09:08:04.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588645+0000", 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588645+0000", 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588644+0000", 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588650+0000", 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588654+0000", 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1779999999999996e-06 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588658+0000", 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4319999999999999e-06 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72053 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596511+0000", 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9967299, 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001768667, 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72053 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596511+0000", 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596511+0000", 2024-09-13T09:08:04.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596510+0000", 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596512+0000", 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8610000000000001e-06 2024-09-13T09:08:04.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596518+0000", 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2830000000000001e-06 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596521+0000", 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3670000000000001e-06 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.209 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71499 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566695+0000", 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026546114999999999, 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031617025, 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.210 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71499 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566695+0000", 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566695+0000", 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566694+0000", 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566699+0000", 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.702e-06 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566712+0000", 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3338e-05 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566740+0000", 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8455000000000001e-05 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71554 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567866+0000", 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025375108, 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030476878999999998, 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.214 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71554 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567866+0000", 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567866+0000", 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567865+0000", 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567868+0000", 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2540000000000001e-06 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567875+0000", 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5220000000000004e-06 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567880+0000", 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8960000000000002e-06 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71618 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568862+0000", 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024378844, 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029517865000000001, 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.218 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71618 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568862+0000", 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568862+0000", 2024-09-13T09:08:04.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568861+0000", 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568866+0000", 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3259999999999997e-06 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568881+0000", 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5401999999999999e-05 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568894+0000", 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2678e-05 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.221 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71850 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569936+0000", 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023304293, 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028473136999999999, 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.222 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71850 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569936+0000", 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569936+0000", 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569936+0000", 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569941+0000", 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1259999999999997e-06 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569965+0000", 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4150000000000001e-05 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569985+0000", 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9825000000000001e-05 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71937 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576592+0000", 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016648990999999998, 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021849184000000001, 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.226 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71937 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576592+0000", 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576592+0000", 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576591+0000", 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576593+0000", 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.685e-06 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576597+0000", 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1969999999999998e-06 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576601+0000", 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3720000000000001e-06 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.229 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71970 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580683+0000", 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012557666, 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017786949, 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71970 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580683+0000", 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580683+0000", 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580683+0000", 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580684+0000", 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6780000000000001e-06 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580688+0000", 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.019e-06 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580692+0000", 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3440000000000001e-06 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72022 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588661+0000", 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0045799270000000001, 2024-09-13T09:08:04.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0098412770000000007, 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72022 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588661+0000", 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588661+0000", 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588660+0000", 2024-09-13T09:08:04.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588662+0000", 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7e-06 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588666+0000", 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1189999999999999e-06 2024-09-13T09:08:04.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588670+0000", 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4699999999999998e-06 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72054 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.237 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596524+0000", 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.996717, 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020080779999999999, 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72054 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596524+0000", 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596524+0000", 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596523+0000", 2024-09-13T09:08:04.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596525+0000", 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.683e-06 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596529+0000", 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0570000000000003e-06 2024-09-13T09:08:04.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596533+0000", 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4790000000000001e-06 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71501 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.241 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566752+0000", 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026489024, 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031809381999999997, 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71501 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566752+0000", 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566752+0000", 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566751+0000", 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566755+0000", 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4900000000000002e-06 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566767+0000", 2024-09-13T09:08:04.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2088e-05 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566782+0000", 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4493e-05 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71556 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567883+0000", 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025357347999999998, 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030709630000000002, 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71556 2024-09-13T09:08:04.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567883+0000", 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567883+0000", 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567883+0000", 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567891+0000", 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6399999999999997e-06 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567896+0000", 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5759999999999996e-06 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567901+0000", 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4870000000000004e-06 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71788 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568902+0000", 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024338763999999999, 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029721279, 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71788 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.250 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568902+0000", 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568902+0000", 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568901+0000", 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568906+0000", 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4780000000000002e-06 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568922+0000", 2024-09-13T09:08:04.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5869e-05 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568932+0000", 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0138e-05 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71852 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.253 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569995+0000", 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023245208, 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028659527000000001, 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71852 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569995+0000", 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569995+0000", 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569995+0000", 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569999+0000", 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570017+0000", 2024-09-13T09:08:04.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7904e-05 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570027+0000", 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6379999999999999e-06 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.257 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71938 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576604+0000", 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016637123, 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022081727999999998, 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71938 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576604+0000", 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576604+0000", 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576603+0000", 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576605+0000", 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.686e-06 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576609+0000", 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1479999999999999e-06 2024-09-13T09:08:04.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576612+0000", 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3220000000000001e-06 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71972 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.580695+0000", 2024-09-13T09:08:04.261 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.012546125999999999, 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.018026121999999999, 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71972 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580695+0000", 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580695+0000", 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580694+0000", 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580696+0000", 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7430000000000001e-06 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580700+0000", 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0899999999999998e-06 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580704+0000", 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.461e-06 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72023 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588673+0000", 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0045678259999999997, 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010083257, 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.265 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72023 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588673+0000", 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588673+0000", 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588673+0000", 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588674+0000", 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.666e-06 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588678+0000", 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1479999999999999e-06 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588682+0000", 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4769999999999999e-06 2024-09-13T09:08:04.268 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72055 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596536+0000", 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9967051, 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002252446, 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.269 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72055 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596536+0000", 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596536+0000", 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596535+0000", 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596537+0000", 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6139999999999999e-06 2024-09-13T09:08:04.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596547+0000", 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8970000000000003e-06 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596551+0000", 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7129999999999999e-06 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71503 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566823+0000", 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026417603000000001, 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031995584000000001, 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71503 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.273 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566823+0000", 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566823+0000", 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566823+0000", 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566826+0000", 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.05e-06 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566834+0000", 2024-09-13T09:08:04.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4549999999999995e-06 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566842+0000", 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1580000000000005e-06 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.276 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71558 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567905+0000", 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025336127, 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030945232999999999, 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71558 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567905+0000", 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567905+0000", 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567904+0000", 2024-09-13T09:08:04.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567906+0000", 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4059999999999999e-06 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567912+0000", 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3179999999999997e-06 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567916+0000", 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5959999999999997e-06 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71789 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568976+0000", 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024264928000000002, 2024-09-13T09:08:04.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029905761999999999, 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71789 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568976+0000", 2024-09-13T09:08:04.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568976+0000", 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568975+0000", 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568979+0000", 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.49e-06 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568987+0000", 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568992+0000", 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4160000000000003e-06 2024-09-13T09:08:04.283 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71855 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570037+0000", 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023203713000000001, 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028876189, 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.284 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71855 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570037+0000", 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570037+0000", 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570036+0000", 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570040+0000", 2024-09-13T09:08:04.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9820000000000002e-06 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570052+0000", 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1976e-05 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570064+0000", 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1874e-05 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.287 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71939 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576615+0000", 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016625219, 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022326895999999999, 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71939 2024-09-13T09:08:04.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576615+0000", 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576615+0000", 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576615+0000", 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576630+0000", 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5087e-05 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576636+0000", 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2669999999999997e-06 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576639+0000", 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7780000000000001e-06 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71974 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584461+0000", 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0087796560000000003, 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014510334, 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71974 2024-09-13T09:08:04.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584461+0000", 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584461+0000", 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.580707+0000", 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9962459 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584465+0000", 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037583130000000001 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584477+0000", 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1392999999999999e-05 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584482+0000", 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72024 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588686+0000", 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0045544699999999997, 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010315980000000001, 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.296 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72024 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588686+0000", 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588686+0000", 2024-09-13T09:08:04.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588686+0000", 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588688+0000", 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7379999999999999e-06 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588692+0000", 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588695+0000", 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.36e-06 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72056 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596554+0000", 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9966869, 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024778140000000001, 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72056 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.300 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596554+0000", 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596554+0000", 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596553+0000", 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596555+0000", 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.686e-06 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596559+0000", 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3069999999999996e-06 2024-09-13T09:08:04.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596563+0000", 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4170000000000001e-06 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71505 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566850+0000", 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.02639035, 2024-09-13T09:08:04.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032213712999999998, 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71505 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566850+0000", 2024-09-13T09:08:04.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566850+0000", 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566850+0000", 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566854+0000", 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2780000000000004e-06 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566865+0000", 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0868e-05 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566875+0000", 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0706e-05 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71560 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567920+0000", 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025320537000000001, 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031174611000000001, 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.307 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71560 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567920+0000", 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567920+0000", 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567920+0000", 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567922+0000", 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.21e-06 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567927+0000", 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2839999999999997e-06 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567932+0000", 2024-09-13T09:08:04.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4139999999999996e-06 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71790 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568997+0000", 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024244103999999999, 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030135255, 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.311 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71790 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568997+0000", 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568997+0000", 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568996+0000", 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568998+0000", 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3259999999999998e-06 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569004+0000", 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3519999999999997e-06 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569008+0000", 2024-09-13T09:08:04.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5059999999999997e-06 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71857 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570096+0000", 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023144741999999999, 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029065507000000001, 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.315 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71857 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570096+0000", 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570096+0000", 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570095+0000", 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570099+0000", 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3139999999999999e-06 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570107+0000", 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9039999999999992e-06 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570112+0000", 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:08:04.318 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71940 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576643+0000", 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016597908000000001, 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022551003, 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.319 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71940 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576643+0000", 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576643+0000", 2024-09-13T09:08:04.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576642+0000", 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576644+0000", 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576649+0000", 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2470000000000002e-06 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576652+0000", 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3639999999999999e-06 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71976 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584499+0000", 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0087421080000000002, 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014725748, 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71976 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584499+0000", 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584499+0000", 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584498+0000", 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584500+0000", 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2699999999999999e-06 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584505+0000", 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9309999999999999e-06 2024-09-13T09:08:04.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584509+0000", 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.817e-06 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72025 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.326 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588698+0000", 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0045425800000000001, 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010558865000000001, 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72025 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588698+0000", 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588698+0000", 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588698+0000", 2024-09-13T09:08:04.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588700+0000", 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.905e-06 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588704+0000", 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0849999999999999e-06 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588707+0000", 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3969999999999998e-06 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72057 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596567+0000", 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9966741, 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027197549999999999, 2024-09-13T09:08:04.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72057 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596567+0000", 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596567+0000", 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596566+0000", 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596572+0000", 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6180000000000003e-06 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596576+0000", 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4100000000000001e-06 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596580+0000", 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.315e-06 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.333 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71507 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566883+0000", 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026357774, 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032434047000000001, 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.334 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71507 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566883+0000", 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566883+0000", 2024-09-13T09:08:04.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566882+0000", 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566886+0000", 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7469999999999999e-06 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566907+0000", 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1338000000000001e-05 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566920+0000", 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2687999999999999e-05 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71562 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567936+0000", 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025305102999999999, 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031410887999999998, 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71562 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567936+0000", 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567936+0000", 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567935+0000", 2024-09-13T09:08:04.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567937+0000", 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2180000000000002e-06 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567943+0000", 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2639999999999999e-06 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567947+0000", 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4059999999999998e-06 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71791 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569012+0000", 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024228546, 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030363411999999999, 2024-09-13T09:08:04.341 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71791 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569012+0000", 2024-09-13T09:08:04.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569012+0000", 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569012+0000", 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569014+0000", 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569019+0000", 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3519999999999997e-06 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569024+0000", 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5390000000000001e-06 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71862 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570116+0000", 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023124387999999999, 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029291632000000001, 2024-09-13T09:08:04.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71862 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570116+0000", 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570116+0000", 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570116+0000", 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570118+0000", 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.446e-06 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570124+0000", 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2660000000000001e-06 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570128+0000", 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4719999999999997e-06 2024-09-13T09:08:04.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71941 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576669+0000", 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016571289999999999, 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022769286, 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.349 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71941 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576669+0000", 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576669+0000", 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576669+0000", 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576671+0000", 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1629999999999999e-06 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576679+0000", 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6249999999999998e-06 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576682+0000", 2024-09-13T09:08:04.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.749e-06 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71978 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584512+0000", 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0087284110000000002, 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014962593999999999, 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.353 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71978 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584512+0000", 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584512+0000", 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584512+0000", 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584514+0000", 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0370000000000001e-06 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584524+0000", 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9029999999999999e-06 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584529+0000", 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2009999999999998e-06 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.357 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72026 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588710+0000", 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0045305989999999997, 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010794453000000001, 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72026 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.358 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588710+0000", 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588710+0000", 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588710+0000", 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588711+0000", 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5740000000000001e-06 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588716+0000", 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1999999999999996e-06 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588719+0000", 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4410000000000002e-06 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.361 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72058 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596583+0000", 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9966583, 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029534169999999998, 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72058 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596583+0000", 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596583+0000", 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596582+0000", 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596584+0000", 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.635e-06 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596588+0000", 2024-09-13T09:08:04.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1119999999999998e-06 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596592+0000", 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0999999999999997e-06 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71509 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.365 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566932+0000", 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026308643, 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032633658000000003, 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71509 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566932+0000", 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566932+0000", 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566931+0000", 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566936+0000", 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4399999999999998e-06 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566943+0000", 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1470000000000004e-06 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566948+0000", 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9320000000000004e-06 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71564 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567951+0000", 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025289810999999999, 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031646096999999998, 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.370 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71564 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567951+0000", 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567951+0000", 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567951+0000", 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567953+0000", 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2500000000000001e-06 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567958+0000", 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2970000000000003e-06 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567963+0000", 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.454e-06 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71792 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569027+0000", 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024213242999999999, 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030600768, 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.374 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71792 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569027+0000", 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569027+0000", 2024-09-13T09:08:04.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569027+0000", 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569046+0000", 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.908e-05 2024-09-13T09:08:04.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569053+0000", 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7560000000000002e-06 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569058+0000", 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6360000000000002e-06 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.377 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71864 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570133+0000", 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023107375999999999, 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029525592, 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71864 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570133+0000", 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570133+0000", 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570133+0000", 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570135+0000", 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4940000000000002e-06 2024-09-13T09:08:04.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570141+0000", 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3600000000000004e-06 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570145+0000", 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5839999999999996e-06 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.381 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71942 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576694+0000", 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016546716, 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022994067999999999, 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.382 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71942 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576694+0000", 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576694+0000", 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576694+0000", 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576696+0000", 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8419999999999999e-06 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576700+0000", 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7559999999999999e-06 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576704+0000", 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.659e-06 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71980 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584532+0000", 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0087084580000000005, 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.015187541000000001, 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71980 2024-09-13T09:08:04.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584532+0000", 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584532+0000", 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584532+0000", 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584534+0000", 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.99e-06 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584541+0000", 2024-09-13T09:08:04.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8469999999999998e-06 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584544+0000", 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.517e-06 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72027 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.389 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588722+0000", 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0045187860000000003, 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011029425000000001, 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72027 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588722+0000", 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588722+0000", 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588722+0000", 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588723+0000", 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.677e-06 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588731+0000", 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3300000000000001e-06 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588736+0000", 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9320000000000004e-06 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72059 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596595+0000", 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9966455, 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003186796, 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72059 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.394 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596595+0000", 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596595+0000", 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596595+0000", 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596596+0000", 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.646e-06 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596601+0000", 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0759999999999996e-06 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596604+0000", 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3440000000000001e-06 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71511 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566952+0000", 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026288369999999998, 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032862258999999998, 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71511 2024-09-13T09:08:04.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566952+0000", 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566952+0000", 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566952+0000", 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566954+0000", 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5380000000000001e-06 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566960+0000", 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7819999999999999e-06 2024-09-13T09:08:04.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566968+0000", 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2139999999999996e-06 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71566 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567966+0000", 2024-09-13T09:08:04.401 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025274517999999999, 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031877863999999999, 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71566 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567966+0000", 2024-09-13T09:08:04.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567966+0000", 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567966+0000", 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567968+0000", 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0940000000000002e-06 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567973+0000", 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2599999999999996e-06 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567978+0000", 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3900000000000003e-06 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71793 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569062+0000", 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024178655, 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030819172999999998, 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.405 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71793 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569062+0000", 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569062+0000", 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569062+0000", 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569064+0000", 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.486e-06 2024-09-13T09:08:04.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569070+0000", 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4060000000000004e-06 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569074+0000", 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6800000000000001e-06 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.408 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71866 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570149+0000", 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023091641, 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029761761000000001, 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71866 2024-09-13T09:08:04.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570149+0000", 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570149+0000", 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570149+0000", 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570151+0000", 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1280000000000002e-06 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570156+0000", 2024-09-13T09:08:04.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3480000000000003e-06 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570161+0000", 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5000000000000001e-06 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71943 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.412 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576707+0000", 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016533368999999999, 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023234804000000001, 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71943 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576707+0000", 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576707+0000", 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576707+0000", 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576709+0000", 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.784e-06 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576719+0000", 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0550000000000001e-05 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576729+0000", 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3579999999999992e-06 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71982 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584548+0000", 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00869259, 2024-09-13T09:08:04.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.015423027000000001, 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71982 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584548+0000", 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584548+0000", 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584548+0000", 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584550+0000", 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.764e-06 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584554+0000", 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2660000000000003e-06 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584558+0000", 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0330000000000002e-06 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72028 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588738+0000", 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0045022049999999996, 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.01126506, 2024-09-13T09:08:04.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72028 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588738+0000", 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588738+0000", 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588738+0000", 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588740+0000", 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6789999999999999e-06 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588744+0000", 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1139999999999999e-06 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588747+0000", 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.444e-06 2024-09-13T09:08:04.423 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72362 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596607+0000", 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.996634, 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0034259910000000002, 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.424 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72362 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596607+0000", 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596607+0000", 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596606+0000", 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596608+0000", 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5629999999999999e-06 2024-09-13T09:08:04.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596615+0000", 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4410000000000004e-06 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596619+0000", 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3670000000000001e-06 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71513 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.566976+0000", 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.02626502, 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.033088187999999998, 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71513 2024-09-13T09:08:04.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566976+0000", 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566976+0000", 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566975+0000", 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566979+0000", 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8240000000000001e-06 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.566990+0000", 2024-09-13T09:08:04.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1094000000000001e-05 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567000+0000", 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0477e-05 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71568 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.431 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567981+0000", 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025259288000000001, 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032111681000000003, 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71568 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567981+0000", 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567981+0000", 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567981+0000", 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567983+0000", 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1880000000000001e-06 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567988+0000", 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.288e-06 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567993+0000", 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6e-06 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71794 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569078+0000", 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024162550000000001, 2024-09-13T09:08:04.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031047332, 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71794 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569078+0000", 2024-09-13T09:08:04.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569078+0000", 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569078+0000", 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569080+0000", 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1550000000000001e-06 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569085+0000", 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3399999999999997e-06 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569090+0000", 2024-09-13T09:08:04.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4859999999999999e-06 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71868 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570164+0000", 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023076402999999999, 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029990823, 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.439 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71868 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570164+0000", 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570164+0000", 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570164+0000", 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570166+0000", 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3420000000000002e-06 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570172+0000", 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3680000000000001e-06 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570176+0000", 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3919999999999996e-06 2024-09-13T09:08:04.442 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71944 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576732+0000", 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016508521000000002, 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023453841, 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.443 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71944 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576732+0000", 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576732+0000", 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576732+0000", 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576734+0000", 2024-09-13T09:08:04.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9199999999999998e-06 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576738+0000", 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4800000000000003e-06 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576742+0000", 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4379999999999999e-06 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.446 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71984 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584573+0000", 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0086678729999999996, 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.015644740000000001, 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71984 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584573+0000", 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584573+0000", 2024-09-13T09:08:04.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584572+0000", 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584575+0000", 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4559999999999999e-06 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584580+0000", 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0200000000000002e-06 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584584+0000", 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8380000000000004e-06 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72029 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.588750+0000", 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0044902539999999999, 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011502010999999999, 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72029 2024-09-13T09:08:04.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588750+0000", 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588750+0000", 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588750+0000", 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588752+0000", 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.759e-06 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588758+0000", 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4729999999999997e-06 2024-09-13T09:08:04.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588762+0000", 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.427e-06 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72364 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.454 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596622+0000", 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9966192, 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036638410000000001, 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72364 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596622+0000", 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596622+0000", 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596621+0000", 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596623+0000", 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.646e-06 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596628+0000", 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4070000000000001e-06 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596632+0000", 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4170000000000001e-06 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71515 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567009+0000", 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026231915000000001, 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.033306173000000001, 2024-09-13T09:08:04.458 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71515 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567009+0000", 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567009+0000", 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567008+0000", 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567012+0000", 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7579999999999998e-06 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567023+0000", 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1260000000000001e-05 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567034+0000", 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0742e-05 2024-09-13T09:08:04.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71570 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567997+0000", 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025243735999999999, 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032348913, 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.462 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71570 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567997+0000", 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567997+0000", 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567997+0000", 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567999+0000", 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1840000000000002e-06 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568005+0000", 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2620000000000004e-06 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568009+0000", 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4340000000000002e-06 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71795 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569093+0000", 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024147312000000001, 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031285107999999999, 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71795 2024-09-13T09:08:04.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569093+0000", 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569093+0000", 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569093+0000", 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569095+0000", 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.154e-06 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569101+0000", 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3820000000000003e-06 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569105+0000", 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4159999999999997e-06 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71870 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570181+0000", 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023059672999999999, 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.03024044, 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71870 2024-09-13T09:08:04.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570181+0000", 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570181+0000", 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570181+0000", 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570183+0000", 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1059999999999998e-06 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570188+0000", 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.412e-06 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570193+0000", 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3660000000000002e-06 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71945 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576745+0000", 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016495830999999999, 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023706008000000001, 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.474 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71945 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576745+0000", 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576745+0000", 2024-09-13T09:08:04.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576745+0000", 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576746+0000", 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.688e-06 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576752+0000", 2024-09-13T09:08:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3959999999999997e-06 2024-09-13T09:08:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576755+0000", 2024-09-13T09:08:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.444e-06 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71985 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584587+0000", 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0086537839999999994, 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.015896521, 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71985 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584587+0000", 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584587+0000", 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584587+0000", 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584588+0000", 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6789999999999999e-06 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584592+0000", 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0799999999999999e-06 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584596+0000", 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.427e-06 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72030 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592515+0000", 2024-09-13T09:08:04.482 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00072526800000000005, 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0079971209999999994, 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72030 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592515+0000", 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592515+0000", 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.588764+0000", 2024-09-13T09:08:04.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9962487 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592519+0000", 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037547589999999999 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592528+0000", 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2390000000000008e-06 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592534+0000", 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5670000000000002e-06 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72366 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596649+0000", 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9965911, 2024-09-13T09:08:04.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038940509999999999, 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72366 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596649+0000", 2024-09-13T09:08:04.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596649+0000", 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596649+0000", 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596651+0000", 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.086e-06 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596657+0000", 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0959999999999997e-06 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596662+0000", 2024-09-13T09:08:04.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71517 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567082+0000", 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026158758000000001, 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.033490985000000001, 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.490 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71517 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567082+0000", 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567082+0000", 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567081+0000", 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567087+0000", 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7039999999999999e-06 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567103+0000", 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6186000000000001e-05 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567115+0000", 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2163e-05 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71572 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568014+0000", 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025227198999999999, 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032589025000000001, 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71572 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568014+0000", 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568014+0000", 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568013+0000", 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568015+0000", 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2620000000000001e-06 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568022+0000", 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2700000000000001e-06 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568026+0000", 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7299999999999996e-06 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71796 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569109+0000", 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024131837999999999, 2024-09-13T09:08:04.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031528566000000001, 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71796 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569109+0000", 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569109+0000", 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569108+0000", 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569111+0000", 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.232e-06 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569116+0000", 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4600000000000002e-06 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569124+0000", 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9160000000000001e-06 2024-09-13T09:08:04.500 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71873 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570196+0000", 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023044313, 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030470476, 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.501 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71873 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570196+0000", 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570196+0000", 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570196+0000", 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570198+0000", 2024-09-13T09:08:04.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.232e-06 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570204+0000", 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3709999999999999e-06 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570208+0000", 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6399999999999996e-06 2024-09-13T09:08:04.504 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71946 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576758+0000", 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016482594, 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023939566999999998, 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.505 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71946 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576758+0000", 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576758+0000", 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576758+0000", 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576759+0000", 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.68e-06 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576764+0000", 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.228e-06 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576767+0000", 2024-09-13T09:08:04.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.693e-06 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71987 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584600+0000", 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0086409899999999994, 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016127071999999999, 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.509 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71987 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584600+0000", 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584600+0000", 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584599+0000", 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584601+0000", 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.714e-06 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584610+0000", 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5110000000000003e-06 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584626+0000", 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6127e-05 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.512 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72031 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592552+0000", 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00068864499999999999, 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0082069459999999997, 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.513 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72031 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592552+0000", 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592552+0000", 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592552+0000", 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592554+0000", 2024-09-13T09:08:04.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.368e-06 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592559+0000", 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592563+0000", 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0999999999999997e-06 2024-09-13T09:08:04.516 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72368 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596665+0000", 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9965754, 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0041231740000000003, 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.517 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72368 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596665+0000", 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596665+0000", 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596665+0000", 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596667+0000", 2024-09-13T09:08:04.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.657e-06 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596671+0000", 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0969999999999999e-06 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596675+0000", 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.805e-06 2024-09-13T09:08:04.520 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71519 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567125+0000", 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026115218999999999, 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.033693920000000002, 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.521 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71519 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567125+0000", 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567125+0000", 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567125+0000", 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567129+0000", 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2320000000000003e-06 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567143+0000", 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4025999999999999e-05 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567162+0000", 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9170000000000001e-05 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71574 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568031+0000", 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025210135000000002, 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032818193000000002, 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.525 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71574 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568031+0000", 2024-09-13T09:08:04.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568031+0000", 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568030+0000", 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568033+0000", 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568044+0000", 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0752e-05 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568055+0000", 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0446e-05 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71797 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569134+0000", 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024106786000000002, 2024-09-13T09:08:04.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031748390000000001, 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71797 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569134+0000", 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569134+0000", 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569133+0000", 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569137+0000", 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0609999999999997e-06 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569162+0000", 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4664e-05 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569173+0000", 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1488000000000001e-05 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71878 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570212+0000", 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023028541999999999, 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.030711203999999999, 2024-09-13T09:08:04.533 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71878 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570212+0000", 2024-09-13T09:08:04.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570212+0000", 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570212+0000", 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570219+0000", 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3660000000000003e-06 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570225+0000", 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4779999999999999e-06 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570229+0000", 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5859999999999998e-06 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71947 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576770+0000", 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016470339000000001, 2024-09-13T09:08:04.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.024224422999999998, 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71947 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576770+0000", 2024-09-13T09:08:04.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576770+0000", 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576770+0000", 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576772+0000", 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7999999999999999e-06 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576779+0000", 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8460000000000002e-06 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576782+0000", 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.507e-06 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71989 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584630+0000", 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0086110200000000005, 2024-09-13T09:08:04.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016426975, 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71989 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584630+0000", 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584630+0000", 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584629+0000", 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584631+0000", 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.995e-06 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584638+0000", 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584641+0000", 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5240000000000001e-06 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72032 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592566+0000", 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00067423099999999996, 2024-09-13T09:08:04.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0085502719999999994, 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72032 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592566+0000", 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592566+0000", 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592566+0000", 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592568+0000", 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.815e-06 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592572+0000", 2024-09-13T09:08:04.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2429999999999999e-06 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592576+0000", 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.58e-06 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.549 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72370 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596677+0000", 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9965634, 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044964269999999999, 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72370 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.550 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596677+0000", 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596677+0000", 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596677+0000", 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596679+0000", 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8530000000000001e-06 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596683+0000", 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1629999999999998e-06 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596686+0000", 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.27e-06 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71521 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567214+0000", 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.026026614, 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.034001387000000001, 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71521 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.554 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567214+0000", 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567214+0000", 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567213+0000", 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567218+0000", 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8420000000000004e-06 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567230+0000", 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2497000000000001e-05 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567240+0000", 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5820000000000008e-06 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.557 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71576 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568076+0000", 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025164929999999999, 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.033173486000000002, 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.558 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71576 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568076+0000", 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568076+0000", 2024-09-13T09:08:04.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568075+0000", 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568080+0000", 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5719999999999996e-06 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568094+0000", 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076e-05 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568105+0000", 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1042e-05 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71798 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569182+0000", 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024059183000000001, 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032097373999999998, 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71798 2024-09-13T09:08:04.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569182+0000", 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569182+0000", 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569181+0000", 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569185+0000", 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2699999999999998e-06 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.564 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569199+0000", 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3585e-05 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569213+0000", 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.42e-05 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71880 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570233+0000", 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023007429999999999, 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031076804, 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71880 2024-09-13T09:08:04.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570233+0000", 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570233+0000", 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570233+0000", 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570235+0000", 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570241+0000", 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4940000000000003e-06 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570247+0000", 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0340000000000002e-06 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71948 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576785+0000", 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016455335000000001, 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.024556101, 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.570 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71948 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576785+0000", 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576785+0000", 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576785+0000", 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576787+0000", 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.747e-06 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576791+0000", 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1239999999999998e-06 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576794+0000", 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4259999999999999e-06 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71991 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584644+0000", 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0085967479999999995, 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016762467999999999, 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.574 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71991 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584644+0000", 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584644+0000", 2024-09-13T09:08:04.575 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584644+0000", 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584645+0000", 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7230000000000001e-06 2024-09-13T09:08:04.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584652+0000", 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8040000000000004e-06 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584657+0000", 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3379999999999998e-06 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.577 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72033 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592579+0000", 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00066196799999999997, 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0088880250000000008, 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.578 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72033 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592579+0000", 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592579+0000", 2024-09-13T09:08:04.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592578+0000", 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592584+0000", 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8359999999999997e-06 2024-09-13T09:08:04.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592589+0000", 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3250000000000001e-06 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592592+0000", 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5719999999999999e-06 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.581 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72372 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596690+0000", 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9965506, 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048417139999999996, 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.582 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72372 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596690+0000", 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596690+0000", 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596690+0000", 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596691+0000", 2024-09-13T09:08:04.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7829999999999999e-06 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596696+0000", 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1389999999999997e-06 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596699+0000", 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:04.585 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71523 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.567249+0000", 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025992099000000001, 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.034334794000000002, 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.586 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71523 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567249+0000", 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567249+0000", 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567248+0000", 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567251+0000", 2024-09-13T09:08:04.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2820000000000001e-06 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567260+0000", 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5779999999999999e-06 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.567265+0000", 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0359999999999997e-06 2024-09-13T09:08:04.589 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71578 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.568113+0000", 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025128184000000001, 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.033515034999999999, 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.590 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71578 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568113+0000", 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568113+0000", 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568112+0000", 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568116+0000", 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.002e-06 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568145+0000", 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8807e-05 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.568157+0000", 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1986e-05 2024-09-13T09:08:04.593 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71799 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.569224+0000", 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024016740000000002, 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.032434561000000001, 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.594 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71799 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569224+0000", 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569224+0000", 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569223+0000", 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569228+0000", 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6380000000000003e-06 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569251+0000", 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3728999999999999e-05 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.569263+0000", 2024-09-13T09:08:04.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1902000000000001e-05 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71883 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.570266+0000", 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.022974977000000001, 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.031422529999999997, 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.598 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71883 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570266+0000", 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570266+0000", 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570265+0000", 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570269+0000", 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5379999999999996e-06 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570281+0000", 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1946e-05 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.570290+0000", 2024-09-13T09:08:04.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0899999999999994e-06 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71949 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.576797+0000", 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016443541999999998, 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.024923539000000001, 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.602 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71949 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576797+0000", 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576797+0000", 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576797+0000", 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576799+0000", 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7460000000000001e-06 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576803+0000", 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0879999999999997e-06 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.576806+0000", 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3060000000000002e-06 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.605 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:71993 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.584660+0000", 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0085808989999999995, 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017097324000000001, 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.606 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 71993 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584660+0000", 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584660+0000", 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584660+0000", 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584661+0000", 2024-09-13T09:08:04.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.765e-06 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584667+0000", 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4550000000000003e-06 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.584670+0000", 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4240000000000002e-06 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.609 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72034 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.592595+0000", 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00064558200000000001, 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0092238879999999995, 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.610 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72034 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592595+0000", 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592595+0000", 2024-09-13T09:08:04.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592595+0000", 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592597+0000", 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6869999999999999e-06 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592601+0000", 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1169999999999997e-06 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.592604+0000", 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.467e-06 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.613 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:72373 3.1b 3.68c5f63b (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e60)", 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:03.596702+0000", 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9965391, 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0051780960000000001, 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 72373 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596702+0000", 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596702+0000", 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596701+0000", 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596703+0000", 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7379999999999999e-06 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596707+0000", 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1330000000000001e-06 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:03.596710+0000", 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1599999999999998e-06 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 256 2024-09-13T09:08:04.617 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:04.618 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:08:04.619 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:08:04.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:04.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:08:04.622 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:04.616+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:04.719 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:04.713+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:04.755 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:04.755 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:04.756 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:04.756 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:04.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:04.861 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:04.857+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:04.920 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:04.916+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:04.944 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81710 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3371008~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.531316+0000", 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40959711799999998, 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087743142999999996, 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.945 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81710 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531316+0000", 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531316+0000", 2024-09-13T09:08:04.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531315+0000", 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531321+0000", 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4140000000000002e-06 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531345+0000", 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4394000000000001e-05 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531364+0000", 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9145000000000001e-05 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.618840+0000", 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087475601 2024-09-13T09:08:04.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.618950+0000", 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011059499999999999 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619059+0000", 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108831 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81711 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 2969600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.531384+0000", 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40952952199999998, 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087939059, 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81711 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531384+0000", 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531384+0000", 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531383+0000", 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531389+0000", 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.472e-06 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531407+0000", 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7682000000000001e-05 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531418+0000", 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0909e-05 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619102+0000", 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087684178000000002 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619211+0000", 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108915 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619323+0000", 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111778 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.953 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81712 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3385344~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.531451+0000", 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40946259499999998, 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088135657000000006, 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81712 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.954 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531451+0000", 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531451+0000", 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531449+0000", 2024-09-13T09:08:04.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531455+0000", 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5969999999999999e-06 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531466+0000", 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1188e-05 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531477+0000", 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0628000000000001e-05 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619364+0000", 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087887582000000006 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619475+0000", 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110851 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619586+0000", 2024-09-13T09:08:04.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011106499999999999 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81713 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 2983936~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.531483+0000", 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40943019800000002, 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088360198000000001, 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.958 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81713 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531483+0000", 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531483+0000", 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531482+0000", 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531486+0000", 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.269e-06 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531499+0000", 2024-09-13T09:08:04.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.36e-05 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531521+0000", 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1478999999999999e-05 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619629+0000", 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088108056000000004 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619735+0000", 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105844 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619843+0000", 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108508 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81714 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3399680~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.531527+0000", 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40938580499999999, 2024-09-13T09:08:04.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088577734000000005, 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81714 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531527+0000", 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531527+0000", 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531527+0000", 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531530+0000", 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4740000000000001e-06 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531539+0000", 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4339999999999992e-06 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531547+0000", 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5259999999999993e-06 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619885+0000", 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088338133999999999 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.619993+0000", 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107958 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.620105+0000", 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011176200000000001 2024-09-13T09:08:04.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81715 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 2998272~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.531553+0000", 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.409360263, 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088818572999999998, 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81715 2024-09-13T09:08:04.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531553+0000", 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531553+0000", 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531552+0000", 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531555+0000", 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1200000000000002e-06 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531568+0000", 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2639e-05 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531575+0000", 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.426e-06 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.620149+0000", 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088574101000000002 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.620262+0000", 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011346800000000001 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.620372+0000", 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109404 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81716 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3409920~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.532492+0000", 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40842120300000001, 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088377001999999996, 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81716 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532492+0000", 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532492+0000", 2024-09-13T09:08:04.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.531580+0000", 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9990878 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532503+0000", 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00092378299999999996 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532538+0000", 2024-09-13T09:08:04.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4785e-05 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532563+0000", 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4411000000000001e-05 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.620548+0000", 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087985299000000003 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.620714+0000", 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000166016 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.620869+0000", 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015502800000000001 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81717 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3012608~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.975 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.532582+0000", 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.408331265, 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088667382000000003, 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81717 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532582+0000", 2024-09-13T09:08:04.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532582+0000", 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532581+0000", 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532588+0000", 2024-09-13T09:08:04.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.605e-06 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532627+0000", 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8342e-05 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532641+0000", 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4674e-05 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.620936+0000", 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088295177000000002 2024-09-13T09:08:04.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.621096+0000", 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015969500000000001 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.621249+0000", 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000153177 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81718 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3424256~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.532657+0000", 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40825583500000001, 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.089035550000000005, 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81718 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532657+0000", 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532657+0000", 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532656+0000", 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532663+0000", 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6850000000000004e-06 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532708+0000", 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4671e-05 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.532744+0000", 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5918999999999999e-05 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.621311+0000", 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088567253999999998 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.621533+0000", 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022230700000000001 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.621693+0000", 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015962499999999999 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81726 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3481600~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.537073+0000", 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40384008599999999, 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087737291999999995, 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81726 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537073+0000", 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537073+0000", 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537072+0000", 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537078+0000", 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.857e-06 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537092+0000", 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3688e-05 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537100+0000", 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5939999999999994e-06 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.624493+0000", 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087392094000000003 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.624662+0000", 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016982 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.624810+0000", 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014806199999999999 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81727 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3080192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.537107+0000", 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40380654999999999, 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088037380999999998, 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81727 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.988 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537107+0000", 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537107+0000", 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537106+0000", 2024-09-13T09:08:04.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537109+0000", 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3859999999999999e-06 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537123+0000", 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3570999999999999e-05 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537131+0000", 2024-09-13T09:08:04.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0339999999999997e-06 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.624890+0000", 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087758549000000005 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.625018+0000", 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000128804 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.625144+0000", 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000125597 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81728 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3094528~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.537137+0000", 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40377590699999999, 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088432735999999998, 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81728 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537137+0000", 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537137+0000", 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537137+0000", 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537140+0000", 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.597e-06 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537149+0000", 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3399999999999998e-06 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537155+0000", 2024-09-13T09:08:04.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4169999999999997e-06 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.625244+0000", 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088089088999999995 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.625410+0000", 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000166284 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.625570+0000", 2024-09-13T09:08:04.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159727 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81729 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3495936~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.537162+0000", 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.40375090800000002, 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088797220999999996, 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:04.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81729 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537162+0000", 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537162+0000", 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537162+0000", 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537165+0000", 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2119999999999999e-06 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537178+0000", 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2625e-05 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.537184+0000", 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3740000000000003e-06 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.625637+0000", 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088453407999999997 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.625804+0000", 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016663 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:04.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.625960+0000", 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155501 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81738 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3577856~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.541370+0000", 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.39954319399999999, 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087917887, 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:05.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81738 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541370+0000", 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541370+0000", 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541369+0000", 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541376+0000", 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5400000000000001e-06 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541396+0000", 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0013999999999998e-05 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541409+0000", 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3818999999999999e-05 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629043+0000", 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087633787000000005 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629170+0000", 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012679100000000001 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629288+0000", 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117927 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81739 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3162112~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.541453+0000", 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.39945970400000003, 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088231009999999999, 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:05.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81739 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541453+0000", 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541453+0000", 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541453+0000", 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541459+0000", 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541484+0000", 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4621999999999999e-05 2024-09-13T09:08:05.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541515+0000", 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1588000000000003e-05 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629369+0000", 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087853152000000004 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629532+0000", 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016352399999999999 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629685+0000", 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015236399999999999 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81740 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3592192~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.541532+0000", 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.39938122199999998, 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088462339000000001, 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:05.008 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81740 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541532+0000", 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541532+0000", 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541531+0000", 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541538+0000", 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3019999999999997e-06 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541556+0000", 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7873999999999999e-05 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.541575+0000", 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8428999999999999e-05 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629755+0000", 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088180834999999999 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629875+0000", 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119647 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.629994+0000", 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119215 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81747 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3219456~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.544646+0000", 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.396267431, 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087722126999999997, 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:05.012 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81747 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544646+0000", 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544646+0000", 2024-09-13T09:08:05.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544645+0000", 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544649+0000", 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.399e-06 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544657+0000", 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5830000000000007e-06 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544664+0000", 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7909999999999999e-06 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.632143+0000", 2024-09-13T09:08:05.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087478999000000002 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.632257+0000", 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114215 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.632368+0000", 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110711 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81748 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3649536~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.544670+0000", 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.39624279699999998, 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088107538999999999, 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81748 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:05.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544670+0000", 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544670+0000", 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544670+0000", 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:05.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544673+0000", 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.337e-06 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544694+0000", 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0523e-05 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.544701+0000", 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8170000000000001e-06 2024-09-13T09:08:05.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.632474+0000", 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087773130000000005 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.632596+0000", 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121958 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.632778+0000", 2024-09-13T09:08:05.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018233999999999999 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81769 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3387392~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.548986+0000", 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.39192710600000003, 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087758066999999995, 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:05.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81769 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.548986+0000", 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.548986+0000", 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.548985+0000", 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.548992+0000", 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1319999999999996e-06 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.549014+0000", 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1725e-05 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.549028+0000", 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4144999999999999e-05 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.636499+0000", 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087470874000000004 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.636640+0000", 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000141018 2024-09-13T09:08:05.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.636744+0000", 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104545 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:81771 3.1b 3:dc6fa316:::rbd_data.1124c595d92c.0000000000000000:14 [read 3401728~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:04.549043+0000", 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.39187001300000002, 2024-09-13T09:08:05.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088024268000000003, 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 81771 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.549043+0000", 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.549043+0000", 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.549042+0000", 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.549048+0000", 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.417e-06 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.549095+0000", 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7262000000000003e-05 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.549110+0000", 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4917e-05 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.636824+0000", 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087714062999999995 2024-09-13T09:08:05.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.636944+0000", 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119561 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:04.637067+0000", 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000123905 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.029 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.030 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:05.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:05.031 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:05.025+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:05.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:05.108 INFO:teuthology.orchestra.run.smithi031.stdout:2076 writesame 0x207cc6f thru 0x208abc3 data_size 0x1(0xdf55 bytes) 2024-09-13T09:08:05.134 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:05.128+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:05.220 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:87520 3.1e 3:7a22d209:::rbd_object_map.11245dedf53f:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:05.219924+0000", 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00012736, 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000171115, 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:05.221 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 87520 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:05.219924+0000", 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:05.219924+0000", 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:05.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:05.219922+0000", 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:05.219933+0000", 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1430000000000001e-05 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:05.219954+0000", 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0254999999999998e-05 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:05.219964+0000", 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0377000000000001e-05 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:05.220014+0000", 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0294999999999999e-05 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:05.220083+0000", 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8533000000000003e-05 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.224 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-13T09:08:05.225 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 1 2024-09-13T09:08:05.225 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:05.230 INFO:teuthology.orchestra.run.smithi031.stdout:2077 write 0x5c0a7d7 thru 0x5c158f4 (0xb11e bytes) 2024-09-13T09:08:05.232 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:05.225+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:05.238 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:05.263 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:05.263 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:05.282 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:05.282 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:05.300 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:05.300 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:05.317 INFO:teuthology.orchestra.run.smithi031.stdout:2078 write 0xe9de837 thru 0xe9e5ef0 (0x76ba bytes) 2024-09-13T09:08:05.318 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:05.318 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:05.323 INFO:teuthology.orchestra.run.smithi031.stdout:2079 punch from 0x792b2c4 to 0x79334de, (0x821a bytes) 2024-09-13T09:08:05.323 INFO:teuthology.orchestra.run.smithi031.stdout:2080 read 0x530d458 thru 0x5316ac7 (0x9670 bytes) 2024-09-13T09:08:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:2083 read 0x8c6ccce thru 0x8c6f600 (0x2933 bytes) 2024-09-13T09:08:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:2085 write 0x3c5391b thru 0x3c5c92f (0x9015 bytes) 2024-09-13T09:08:05.332 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:05.329+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:05.346 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:05.346 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:05.376 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:05.376 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:05.394 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:05.394 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:05.415 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:05.415 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:05.417 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:05.417 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:05.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:05.440 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:05.440 INFO:teuthology.orchestra.run.smithi031.stdout:2087 write 0xcf1de2b thru 0xcf2c9b5 (0xeb8b bytes) 2024-09-13T09:08:05.440 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:05.436+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:05.480 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:05.480 INFO:teuthology.orchestra.run.smithi031.stdout:2089 write 0x212bd49 thru 0x2131972 (0x5c2a bytes) 2024-09-13T09:08:05.482 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:05.482 INFO:teuthology.orchestra.run.smithi031.stdout:2091 write 0x90844c thru 0x90a22a (0x1ddf bytes) 2024-09-13T09:08:05.534 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:05.528+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:05.542 INFO:teuthology.orchestra.run.smithi031.stdout:2092 punch from 0xc2bc717 to 0xc2bf94f, (0x3238 bytes) 2024-09-13T09:08:05.542 INFO:teuthology.orchestra.run.smithi031.stdout:2093 punch from 0x308939a to 0x3096b08, (0xd76e bytes) 2024-09-13T09:08:05.542 INFO:teuthology.orchestra.run.smithi031.stdout:2094 write 0x569deeb thru 0x56a2077 (0x418d bytes) 2024-09-13T09:08:05.626 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:05.626 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:05.627 INFO:teuthology.orchestra.run.smithi031.stdout:2096 read 0x6b55d8f thru 0x6b6155d (0xb7cf bytes) 2024-09-13T09:08:05.628 INFO:teuthology.orchestra.run.smithi031.stdout:2097 writesame 0x81fac11 thru 0x82086c5 data_size 0x3(0xdab5 bytes) 2024-09-13T09:08:05.634 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:05.632+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:05.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:05.694 INFO:teuthology.orchestra.run.smithi031.stdout:2098 trunc from 0xe9e5ef1 to 0x25a49af 2024-09-13T09:08:05.735 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:05.732+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:05.780 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:05.780 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:05.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:05.840 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:05.837+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:05.847 INFO:teuthology.orchestra.run.smithi031.stdout:2100 read 0xbd5673 thru 0xbdcaee (0x747c bytes) 2024-09-13T09:08:05.848 INFO:teuthology.orchestra.run.smithi031.stdout:2101 punch from 0x20805f1 to 0x20846a7, (0x40b6 bytes) 2024-09-13T09:08:05.885 INFO:teuthology.orchestra.run.smithi031.stdout:2102 write 0xc10bde0 thru 0xc1137cf (0x79f0 bytes) 2024-09-13T09:08:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:2104 read 0x702cddd thru 0x7032ed5 (0x60f9 bytes) 2024-09-13T09:08:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:2105 write 0x1894fc thru 0x18dd71 (0x4876 bytes) 2024-09-13T09:08:05.934 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:05.934 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:05.936 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:05.933+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:05.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:05.960 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:08:05.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:08:06.028 INFO:teuthology.orchestra.run.smithi031.stdout:2106 read 0x58a6026 thru 0x58b4c50 (0xec2b bytes) 2024-09-13T09:08:06.028 INFO:teuthology.orchestra.run.smithi031.stdout:2107 write 0x96c516e thru 0x96cdd67 (0x8bfa bytes) 2024-09-13T09:08:06.037 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:06.033+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:06.126 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:06.126 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:06.138 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:06.133+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:06.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:06.238 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:06.233+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:06.304 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:06.304 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:06.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:06.339 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:06.336+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:06.440 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:06.437+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:06.483 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:06.483 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:06.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:06.542 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:06.536+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:06.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:08:06.667 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:06.664+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:06.674 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:06.674 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:06.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:06.744 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:06.740+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:06.817 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:06.817 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:06.831 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:06.879 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:06.876+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:06.946 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:06.940+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:06.977 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:08:06.981 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:08:07.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:08:07.002 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:07.003 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:07.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:07.063 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:07.060+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:07.148 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:07.145+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:07.180 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:07.180 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:07.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:07.250 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:07.245+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:07.318 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:07.319 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:07.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:07.351 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:07.349+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:07.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:07.449 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:07.450 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:07.451 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:07.449+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:07.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:07.555 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:07.552+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:07.585 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:07.585 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:07.602 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:07.602 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:07.619 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:07.619 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:07.636 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:07.636 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:07.643 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:07.643 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:07.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:07.643 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:07.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.643 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:07.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.687944818, 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:08:07.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:08:07.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88207 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24698880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235056+0000", 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4054158619999999, 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033071099, 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88207 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235056+0000", 2024-09-13T09:08:07.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235056+0000", 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235055+0000", 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235056+0000", 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1790000000000001e-06 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235060+0000", 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0599999999999999e-06 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235062+0000", 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2869999999999999e-06 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268033+0000", 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032970863000000003 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268083+0000", 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0331999999999999e-05 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268127+0000", 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3577e-05 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.652 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88208 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24723456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235064+0000", 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4054075099999999, 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033196759999999999, 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88208 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235064+0000", 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235064+0000", 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235064+0000", 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235065+0000", 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1000000000000001e-06 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235068+0000", 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:08:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235074+0000", 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268163+0000", 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033088701999999998 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268219+0000", 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5909000000000003e-05 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268261+0000", 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2079999999999997e-05 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.657 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88209 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24748032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.405395226, 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033308870999999997, 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88209 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235077+0000", 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.22e-06 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235080+0000", 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235083+0000", 2024-09-13T09:08:07.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268290+0000", 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033207532999999997 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268342+0000", 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999997e-05 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268385+0000", 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2874999999999999e-05 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88210 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24772608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4053867549999999, 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033438816000000003, 2024-09-13T09:08:07.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88210 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235084+0000", 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235086+0000", 2024-09-13T09:08:07.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235088+0000", 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235091+0000", 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4839999999999998e-06 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268421+0000", 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033329923999999997 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268479+0000", 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8529999999999997e-05 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268523+0000", 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4048999999999998e-05 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88211 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24797184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4053782899999998, 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033554247000000002, 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88211 2024-09-13T09:08:07.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235094+0000", 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2300000000000001e-06 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235097+0000", 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.021e-06 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235099+0000", 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268550+0000", 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033450510000000003 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268593+0000", 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3035999999999999e-05 2024-09-13T09:08:07.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268647+0000", 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4336999999999998e-05 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88212 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24821760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4053699019999999, 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033675415, 2024-09-13T09:08:07.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88212 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235102+0000", 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-13T09:08:07.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235105+0000", 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235108+0000", 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.367e-06 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268686+0000", 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033577810999999999 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268737+0000", 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1134999999999997e-05 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268777+0000", 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0187e-05 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88213 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24846336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.405360699, 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033785911000000002, 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88213 2024-09-13T09:08:07.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235110+0000", 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235112+0000", 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.728e-06 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235119+0000", 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235124+0000", 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268806+0000", 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033682479000000001 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268855+0000", 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999998e-05 2024-09-13T09:08:07.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268897+0000", 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1915000000000001e-05 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88214 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24870912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:07.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4053430640000002, 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033897243000000001, 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88214 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:07.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:07.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235130+0000", 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:08:07.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235139+0000", 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235151+0000", 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1856e-05 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268933+0000", 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033782874999999997 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268986+0000", 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2593000000000003e-05 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269026+0000", 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9546999999999999e-05 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88215 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24895488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4053176770000002, 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033999110999999999, 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88215 2024-09-13T09:08:07.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235153+0000", 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235155+0000", 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8810000000000001e-06 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235159+0000", 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:08:07.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235161+0000", 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269056+0000", 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033895001000000001 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269109+0000", 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269153+0000", 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88216 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24920064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4053079190000002, 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120667, 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88216 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235173+0000", 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8490000000000001e-06 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235178+0000", 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:08:07.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235180+0000", 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269188+0000", 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034007280000000001 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269245+0000", 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6891000000000001e-05 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269284+0000", 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9561999999999998e-05 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88217 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24944640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4052885050000001, 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034230180999999998, 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88217 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235184+0000", 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3120000000000001e-06 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235187+0000", 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:08:07.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235192+0000", 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269313+0000", 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120708 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269363+0000", 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9453999999999998e-05 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269413+0000", 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0531000000000001e-05 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88218 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24969216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.405276948, 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034354619000000003, 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88218 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235195+0000", 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235199+0000", 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235201+0000", 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269450+0000", 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034248860999999999 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269502+0000", 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2519000000000003e-05 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269549+0000", 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6740999999999999e-05 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88219 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24993792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4052684069999999, 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034466610000000002, 2024-09-13T09:08:07.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88219 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235204+0000", 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.184e-06 2024-09-13T09:08:07.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235207+0000", 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235209+0000", 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269577+0000", 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034368528000000002 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269628+0000", 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0241999999999998e-05 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269670+0000", 2024-09-13T09:08:07.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1875999999999997e-05 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88220 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25018368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4052605570000001, 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034591746999999999, 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88220 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235212+0000", 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0839999999999999e-06 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235215+0000", 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9830000000000001e-06 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235217+0000", 2024-09-13T09:08:07.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269707+0000", 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034489989999999998 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269762+0000", 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000003e-05 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269803+0000", 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0181e-05 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88221 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25042944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4052524740000001, 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034709489000000003, 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88221 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235220+0000", 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0640000000000001e-06 2024-09-13T09:08:07.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235223+0000", 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235225+0000", 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269830+0000", 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034604633000000003 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269884+0000", 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067000000000003e-05 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269928+0000", 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4607000000000002e-05 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88222 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25067520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4052443280000002, 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034835487999999998, 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88222 2024-09-13T09:08:07.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235228+0000", 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:08:07.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235231+0000", 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235233+0000", 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269963+0000", 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034730039999999997 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270016+0000", 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2581999999999999e-05 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270063+0000", 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6718000000000002e-05 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88223 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25092096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4052361069999999, 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034953250999999998, 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88223 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:07.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235236+0000", 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.127e-06 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235239+0000", 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0180000000000002e-06 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235241+0000", 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270091+0000", 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034849036 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270137+0000", 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6112999999999997e-05 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270189+0000", 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88224 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25116672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.4052279080000001, 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035071738999999998, 2024-09-13T09:08:07.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88224 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235244+0000", 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-13T09:08:07.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235247+0000", 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7750000000000001e-06 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235249+0000", 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270226+0000", 2024-09-13T09:08:07.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034977019999999998 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270275+0000", 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8791000000000001e-05 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270315+0000", 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9953999999999997e-05 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.735 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88225 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25141248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.405219996, 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035182109000000003, 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88225 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.736 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235252+0000", 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235255+0000", 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-13T09:08:07.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235258+0000", 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270337+0000", 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035079359999999997 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270388+0000", 2024-09-13T09:08:07.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0986000000000003e-05 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270433+0000", 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5581000000000003e-05 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:07.740 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:07.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:07.742 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:07.742 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:07.743 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:07.743 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:07.743 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:07.744 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:07.744 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:07.744 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:07.745 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:07.736+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:07.843 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:07.837+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:07.910 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:07.910 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:07.911 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:07.911 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:07.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:07.971 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:07.968+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:07.982 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:08:08.003 INFO:teuthology.orchestra.run.smithi031.stdout:2108 write 0xc325a9b thru 0xc32f441 (0x99a7 bytes) 2024-09-13T09:08:08.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:08:08.006 INFO:teuthology.orchestra.run.smithi031.stdout:2109 write 0x15929ab thru 0x159aa56 (0x80ac bytes) 2024-09-13T09:08:08.055 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:08.052+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:08.077 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:08.077 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:08.078 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:08.078 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:08.095 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:08.144 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:08.141+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:08.153 INFO:teuthology.orchestra.run.smithi031.stdout:2113 writesame 0x3ac7b45 thru 0x3ac8e00 data_size 0xb(0x12bc bytes) 2024-09-13T09:08:08.158 INFO:teuthology.orchestra.run.smithi031.stdout:2114 write 0xbd2f9d2 thru 0xbd32a23 (0x3052 bytes) 2024-09-13T09:08:08.165 INFO:teuthology.orchestra.run.smithi031.stdout:2115 writesame 0x7b41ce0 thru 0x7b4775f data_size 0x40(0x5a80 bytes) 2024-09-13T09:08:08.171 INFO:teuthology.orchestra.run.smithi031.stdout:2116 read 0xbe2176a thru 0xbe2a89f (0x9136 bytes) 2024-09-13T09:08:08.172 INFO:teuthology.orchestra.run.smithi031.stdout:2117 punch from 0xa26d7a9 to 0xa27845c, (0xacb3 bytes) 2024-09-13T09:08:08.172 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:08.175 INFO:teuthology.orchestra.run.smithi031.stdout:2119 trunc from 0xc32f442 to 0x9bf21d6 2024-09-13T09:08:08.204 INFO:teuthology.orchestra.run.smithi031.stdout:2120 write 0x56dda71 thru 0x56ed950 (0xfee0 bytes) 2024-09-13T09:08:08.226 INFO:teuthology.orchestra.run.smithi031.stdout:2122 trunc from 0x9bf21d6 to 0x2fa585 2024-09-13T09:08:08.245 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:08.241+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:08.262 INFO:teuthology.orchestra.run.smithi031.stdout:2123 trunc from 0x2fa585 to 0x82ed8a8 2024-09-13T09:08:08.270 INFO:teuthology.orchestra.run.smithi031.stdout:2124 writesame 0x2264fb5 thru 0x2266cc6 data_size 0x2(0x1d12 bytes) 2024-09-13T09:08:08.279 INFO:teuthology.orchestra.run.smithi031.stdout:2125 write 0x39dc384 thru 0x39e535a (0x8fd7 bytes) 2024-09-13T09:08:08.297 INFO:teuthology.orchestra.run.smithi031.stdout:2126 writesame 0x8c4bc12 thru 0x8c552b3 data_size 0x3(0x96a2 bytes) 2024-09-13T09:08:08.297 INFO:teuthology.orchestra.run.smithi031.stdout:2127 read 0x81a5b33 thru 0x81a5b85 (0x53 bytes) 2024-09-13T09:08:08.309 INFO:teuthology.orchestra.run.smithi031.stdout:2128 trunc from 0x8c552b4 to 0x5face9 2024-09-13T09:08:08.312 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:08.312 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:08.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:08.312 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:08.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.312 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.312 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:08.312 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972647520999999, 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.317 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972630748, 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972593443000001, 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.322 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:08:08.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972573728999997, 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:08:08.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972557404, 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:08.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:08:08.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:08.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972541538000002, 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:08:08.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:08:08.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972525752999999, 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:08:08.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972506682000002, 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:08:08.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972490135000001, 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:08:08.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972462536000002, 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972445294000003, 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:08.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:08:08.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:08:08.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972414851000003, 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:08.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.367 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972395935999998, 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972378147999997, 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:08:08.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972352923999999, 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972334719999999, 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:08:08.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972314402000002, 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:08:08.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972295854000002, 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:08:08.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972278580000001, 2024-09-13T09:08:08.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:08:08.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.972261125999999, 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:08:08.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.405 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:08.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:08.407 INFO:teuthology.orchestra.run.smithi031.stdout:2129 writesame 0xe207160 thru 0xe20ab2a data_size 0xb(0x39cb bytes) 2024-09-13T09:08:08.407 INFO:teuthology.orchestra.run.smithi031.stdout:2131 punch from 0xdae85f0 to 0xdaeabf0, (0x2600 bytes) 2024-09-13T09:08:08.407 INFO:teuthology.orchestra.run.smithi031.stdout:2132 writesame 0x43b16d2 thru 0x43bd553 data_size 0x5(0xbe82 bytes) 2024-09-13T09:08:08.407 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:08.408 INFO:teuthology.orchestra.run.smithi031.stdout:2136 write 0x363dc1c thru 0x3649994 (0xbd79 bytes) 2024-09-13T09:08:08.408 INFO:teuthology.orchestra.run.smithi031.stdout:2137 writesame 0xe1218ac thru 0xe12a56d data_size 0x2b(0x8cc2 bytes) 2024-09-13T09:08:08.408 INFO:teuthology.orchestra.run.smithi031.stdout:2141 write 0xbf4834b thru 0xbf54593 (0xc249 bytes) 2024-09-13T09:08:08.408 INFO:teuthology.orchestra.run.smithi031.stdout:2142 write 0x65e1727 thru 0x65e4f0e (0x37e8 bytes) 2024-09-13T09:08:08.408 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:08.400+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:08.410 INFO:teuthology.orchestra.run.smithi031.stdout:2143 write 0x4720b2f thru 0x472f036 (0xe508 bytes) 2024-09-13T09:08:08.415 INFO:teuthology.orchestra.run.smithi031.stdout:2147 punch from 0x295681 to 0x2a5105, (0xfa84 bytes) 2024-09-13T09:08:08.432 INFO:teuthology.orchestra.run.smithi031.stdout:2148 trunc from 0xe20ab2b to 0x52076a4 2024-09-13T09:08:08.477 INFO:teuthology.orchestra.run.smithi031.stdout:2151 writesame 0xb233bd1 thru 0xb23bb80 data_size 0x18(0x7fb0 bytes) 2024-09-13T09:08:08.477 INFO:teuthology.orchestra.run.smithi031.stdout:2153 write 0x8a81efc thru 0x8a90ca1 (0xeda6 bytes) 2024-09-13T09:08:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:2154 punch from 0xb02ce53 to 0xb038301, (0xb4ae bytes) 2024-09-13T09:08:08.498 INFO:teuthology.orchestra.run.smithi031.stdout:2155 trunc from 0xb23bb81 to 0x58efcee 2024-09-13T09:08:08.506 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:08.500+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:08.517 INFO:teuthology.orchestra.run.smithi031.stdout:2157 punch from 0x1f1cb5c to 0x1f20c57, (0x40fb bytes) 2024-09-13T09:08:08.517 INFO:teuthology.orchestra.run.smithi031.stdout:2158 read 0x47c88bb thru 0x47d31ea (0xa930 bytes) 2024-09-13T09:08:08.518 INFO:teuthology.orchestra.run.smithi031.stdout:2159 read 0x2c995f3 thru 0x2ca43d5 (0xade3 bytes) 2024-09-13T09:08:08.538 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:08.538 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:08.538 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:08.546 INFO:teuthology.orchestra.run.smithi031.stdout:2161 write 0xea9f699 thru 0xeaa4bc9 (0x5531 bytes) 2024-09-13T09:08:08.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:08.555 INFO:teuthology.orchestra.run.smithi031.stdout:2162 write 0x1d406d2 thru 0x1d44398 (0x3cc7 bytes) 2024-09-13T09:08:08.565 INFO:teuthology.orchestra.run.smithi031.stdout:2163 read 0x772f6c9 thru 0x7735ba2 (0x64da bytes) 2024-09-13T09:08:08.565 INFO:teuthology.orchestra.run.smithi031.stdout:2164 punch from 0x293d23d to 0x2946e90, (0x9c53 bytes) 2024-09-13T09:08:08.576 INFO:teuthology.orchestra.run.smithi031.stdout:2165 trunc from 0xeaa4bca to 0x736e6e4 2024-09-13T09:08:08.604 INFO:teuthology.orchestra.run.smithi031.stdout:2166 writesame 0x74c4333 thru 0x74cbc37 data_size 0x17(0x7905 bytes) 2024-09-13T09:08:08.605 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:08.605 INFO:teuthology.orchestra.run.smithi031.stdout:2168 read 0x6f5a60f thru 0x6f62441 (0x7e33 bytes) 2024-09-13T09:08:08.607 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:08.604+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:08.622 INFO:teuthology.orchestra.run.smithi031.stdout:2169 write 0xb8ad08d thru 0xb8bcce5 (0xfc59 bytes) 2024-09-13T09:08:08.631 INFO:teuthology.orchestra.run.smithi031.stdout:2170 clone 20 order 19 su 2048 sc 9 2024-09-13T09:08:08.677 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:08.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:08.678 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:08.678 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:08.678 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:08.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:08.711 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:08.705+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:08.811 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:08.809+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:08.855 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:08:08.867 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:08:08.867 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.735137718999994, 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.868 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:08:08.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:08:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:08:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:08:08.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:08:08.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.906855539, 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.877 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:08:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:08:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:08:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56973 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4501504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6179504429999998, 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091397946999999993, 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56973 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247918+0000", 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247921+0000", 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:08:08.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247927+0000", 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2389999999999999e-06 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247932+0000", 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339077+0000", 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091144366000000004 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339203+0000", 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126919 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339317+0000", 2024-09-13T09:08:08.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113205 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56975 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4515840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6177566759999991, 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091417921999999999, 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56975 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247936+0000", 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999824 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248117+0000", 2024-09-13T09:08:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180827 2024-09-13T09:08:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248127+0000", 2024-09-13T09:08:08.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248133+0000", 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.19e-06 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339347+0000", 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091213581000000002 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339419+0000", 2024-09-13T09:08:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.203e-05 2024-09-13T09:08:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339530+0000", 2024-09-13T09:08:08.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111359 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56976 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3681280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6177132790000002, 2024-09-13T09:08:08.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091677039000000002, 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56976 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248155+0000", 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248159+0000", 2024-09-13T09:08:08.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248166+0000", 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999999e-06 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248172+0000", 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339558+0000", 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091385818999999993 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339705+0000", 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014693599999999999 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339833+0000", 2024-09-13T09:08:08.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012808200000000001 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56990 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3795968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6145727769999993, 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091546566999999995, 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56990 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251298+0000", 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.041e-06 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251303+0000", 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251319+0000", 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5852e-05 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342591+0000", 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091271605000000006 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342728+0000", 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013662799999999999 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342843+0000", 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115442 2024-09-13T09:08:08.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57022 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4334592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.61449015, 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091691988000000002, 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57022 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251382+0000", 2024-09-13T09:08:08.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5659999999999998e-06 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251391+0000", 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0170000000000002e-06 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251397+0000", 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.908e-06 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342870+0000", 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091473410000000005 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342945+0000", 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4175e-05 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343071+0000", 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012638499999999999 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57026 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5113856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6144672129999993, 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091880383999999996, 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57026 2024-09-13T09:08:08.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251404+0000", 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251410+0000", 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251415+0000", 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2129999999999999e-06 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343098+0000", 2024-09-13T09:08:08.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091683073000000004 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343171+0000", 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3149e-05 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343282+0000", 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110899 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57027 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4348928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6144495699999997, 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092079082000000007, 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57027 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251419+0000", 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251421+0000", 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251427+0000", 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4419999999999997e-06 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251432+0000", 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7620000000000003e-06 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343310+0000", 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091878572000000006 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343383+0000", 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2368999999999998e-05 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343499+0000", 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116142 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.919 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57028 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5128192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6144322249999998, 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092372671000000003, 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57028 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251439+0000", 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1730000000000002e-06 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251445+0000", 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251450+0000", 2024-09-13T09:08:08.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343526+0000", 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092076621999999997 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343605+0000", 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8830000000000002e-05 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343810+0000", 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204544 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6144157010000004, 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:08.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:08:08.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:08:08.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6143359030000006, 2024-09-13T09:08:08.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:08:08.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6140868319999999, 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:08:08.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6137736910000005, 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6133204590000005, 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:08:08.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:08:08.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:08.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6132959450000008, 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:08:08.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6130307160000008, 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:08:08.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6127428669999997, 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:08:08.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57037 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6123066660000003, 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092241045999999993, 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57037 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:08.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253152+0000", 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995894 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253567+0000", 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000414886 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253576+0000", 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6750000000000008e-06 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253582+0000", 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345610+0000", 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092027943000000001 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345682+0000", 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2311999999999995e-05 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345804+0000", 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121332 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6122815629999998, 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:08.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:08.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:08.973 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:08.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:08.976 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:08:08.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:08:08.976 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:08.972+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:09.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:08:09.076 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:09.069+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:09.110 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:09.110 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:09.110 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:09.110 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:09.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:09.217 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:09.213+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:09.278 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:09.272+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:09.287 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:09.287 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:09.288 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:09.288 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:09.288 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:09.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:09.340 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:09.341 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:08:09.358 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-13T09:08:09.064978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":774054702,"num_objects":633,"num_object_clones":160,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":169,"num_read":104651,"num_read_kb":363518,"num_write":7148,"num_write_kb":203466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":47754073,"num_keys_recovered":68,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5508,"ondisk_log_size":5508,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":526828,"kb_used_data":245112,"kb_used_omap":12,"kb_used_meta":281587,"kb_avail":754447892,"statfs":{"total":773094113280,"available":772554641408,"internally_reserved":0,"allocated":250994688,"data_stored":260816630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":288345656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":100419972,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":30017,"num_read_kb":35394,"num_write":247,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001245"},"pg_stats":[{"pgid":"3.1f","version":"64'160","reported_seq":668,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.171013+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:08:08.171013+0000","last_peered":"2024-09-13T09:08:08.171013+0000","last_clean":"2024-09-13T09:08:08.171013+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:08:08.171013+0000","last_undegraded":"2024-09-13T09:08:08.171013+0000","last_fullsized":"2024-09-13T09:08:08.171013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":10845646,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":843,"num_read_kb":4918,"num_write":213,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"64'208","reported_seq":1695,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.209774+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:08:08.209774+0000","last_peered":"2024-09-13T09:08:08.209774+0000","last_clean":"2024-09-13T09:08:08.209774+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:08:08.209774+0000","last_undegraded":"2024-09-13T09:08:08.209774+0000","last_fullsized":"2024-09-13T09:08:08.209774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":37602897,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":2432,"num_read_kb":9910,"num_write":243,"num_write_kb":3848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"60'179","reported_seq":2312,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.669984+0000","last_change":"2024-09-13T09:07:15.879980+0000","last_active":"2024-09-13T09:08:05.669984+0000","last_peered":"2024-09-13T09:08:05.669984+0000","last_clean":"2024-09-13T09:08:05.669984+0000","last_became_active":"2024-09-13T09:07:13.856004+0000","last_became_peered":"2024-09-13T09:07:13.856004+0000","last_unstale":"2024-09-13T09:08:05.669984+0000","last_undegraded":"2024-09-13T09:08:05.669984+0000","last_fullsized":"2024-09-13T09:08:05.669984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:37.046343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":48416107,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":2124,"num_read_kb":25278,"num_write":253,"num_write_kb":13248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"61'85","reported_seq":1507,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.004737+0000","last_change":"2024-09-13T09:07:51.093764+0000","last_active":"2024-09-13T09:08:06.004737+0000","last_peered":"2024-09-13T09:08:06.004737+0000","last_clean":"2024-09-13T09:08:06.004737+0000","last_became_active":"2024-09-13T09:07:50.836531+0000","last_became_peered":"2024-09-13T09:07:50.836531+0000","last_unstale":"2024-09-13T09:08:06.004737+0000","last_undegraded":"2024-09-13T09:08:06.004737+0000","last_fullsized":"2024-09-13T09:08:06.004737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:11.585290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":46102973,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2515,"num_read_kb":22664,"num_write":119,"num_write_kb":10107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":3944384,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"60'268","reported_seq":17538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.434612+0000","last_change":"2024-09-13T09:07:16.108727+0000","last_active":"2024-09-13T09:08:05.434612+0000","last_peered":"2024-09-13T09:08:05.434612+0000","last_clean":"2024-09-13T09:08:05.434612+0000","last_became_active":"2024-09-13T09:07:13.862565+0000","last_became_peered":"2024-09-13T09:07:13.862565+0000","last_unstale":"2024-09-13T09:08:05.434612+0000","last_undegraded":"2024-09-13T09:08:05.434612+0000","last_fullsized":"2024-09-13T09:08:05.434612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:32.305423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":58876863,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":18533,"num_read_kb":25518,"num_write":341,"num_write_kb":16770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"61'180","reported_seq":1945,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-13T09:08:05.963785+0000","last_change":"2024-09-13T09:08:05.964026+0000","last_active":"2024-09-13T09:08:05.963785+0000","last_peered":"2024-09-13T09:08:05.894139+0000","last_clean":"2024-09-13T09:08:05.894139+0000","last_became_active":"2024-09-13T09:07:50.833042+0000","last_became_peered":"2024-09-13T09:07:50.833042+0000","last_unstale":"2024-09-13T09:08:05.963785+0000","last_undegraded":"2024-09-13T09:08:05.963785+0000","last_fullsized":"2024-09-13T09:08:05.963785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:47.212272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":47325778,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":3012,"num_read_kb":13732,"num_write":314,"num_write_kb":16562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":31801546,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"60'93","reported_seq":749,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.175265+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:08:03.175265+0000","last_peered":"2024-09-13T09:08:03.175265+0000","last_clean":"2024-09-13T09:08:03.175265+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:08:03.175265+0000","last_undegraded":"2024-09-13T09:08:03.175265+0000","last_fullsized":"2024-09-13T09:08:03.175265+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":2509931,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":580,"num_read_kb":3310,"num_write":93,"num_write_kb":4753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"59'154","reported_seq":1220,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976447+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:08:06.976447+0000","last_peered":"2024-09-13T09:08:06.976447+0000","last_clean":"2024-09-13T09:08:06.976447+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:08:06.976447+0000","last_undegraded":"2024-09-13T09:08:06.976447+0000","last_fullsized":"2024-09-13T09:08:06.976447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":18577308,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":1714,"num_read_kb":8005,"num_write":153,"num_write_kb":3690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"62'240","reported_seq":2822,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976621+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:08:06.976621+0000","last_peered":"2024-09-13T09:08:06.976621+0000","last_clean":"2024-09-13T09:08:06.976621+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:08:06.976621+0000","last_undegraded":"2024-09-13T09:08:06.976621+0000","last_fullsized":"2024-09-13T09:08:06.976621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":42969160,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3400,"num_read_kb":21858,"num_write":270,"num_write_kb":16106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"61'110","reported_seq":625,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.848566+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:08:05.848566+0000","last_peered":"2024-09-13T09:08:05.848566+0000","last_clean":"2024-09-13T09:08:05.848566+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:08:05.848566+0000","last_undegraded":"2024-09-13T09:08:05.848566+0000","last_fullsized":"2024-09-13T09:08:05.848566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":9332974,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":639,"num_read_kb":3988,"num_write":126,"num_write_kb":3362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"61'205","reported_seq":2005,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.708645+0000","last_change":"2024-09-13T09:07:15.873331+0000","last_active":"2024-09-13T09:08:05.708645+0000","last_peered":"2024-09-13T09:08:05.708645+0000","last_clean":"2024-09-13T09:08:05.708645+0000","last_became_active":"2024-09-13T09:07:13.866813+0000","last_became_peered":"2024-09-13T09:07:13.866813+0000","last_unstale":"2024-09-13T09:08:05.708645+0000","last_undegraded":"2024-09-13T09:08:05.708645+0000","last_fullsized":"2024-09-13T09:08:05.708645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.439602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":45587035,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":2341,"num_read_kb":17771,"num_write":219,"num_write_kb":11683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"61'286","reported_seq":17229,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.058722+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:08:05.058722+0000","last_peered":"2024-09-13T09:08:05.058722+0000","last_clean":"2024-09-13T09:08:05.058722+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:08:05.058722+0000","last_undegraded":"2024-09-13T09:08:05.058722+0000","last_fullsized":"2024-09-13T09:08:05.058722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":16839736,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":5,"num_read":18600,"num_read_kb":25170,"num_write":260,"num_write_kb":6627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944805+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:08:03.944805+0000","last_peered":"2024-09-13T09:08:03.944805+0000","last_clean":"2024-09-13T09:08:03.944805+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:08:03.944805+0000","last_undegraded":"2024-09-13T09:08:03.944805+0000","last_fullsized":"2024-09-13T09:08:03.944805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"56'352","reported_seq":1373,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.178985+0000","last_change":"2024-09-13T09:07:16.108420+0000","last_active":"2024-09-13T09:08:03.178985+0000","last_peered":"2024-09-13T09:08:03.178985+0000","last_clean":"2024-09-13T09:08:03.178985+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:08:03.178985+0000","last_undegraded":"2024-09-13T09:08:03.178985+0000","last_fullsized":"2024-09-13T09:08:03.178985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":15010381,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":2411,"num_read_kb":8412,"num_write":455,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994953+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:08:02.994953+0000","last_peered":"2024-09-13T09:08:02.994953+0000","last_clean":"2024-09-13T09:08:02.994953+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:08:02.994953+0000","last_undegraded":"2024-09-13T09:08:02.994953+0000","last_fullsized":"2024-09-13T09:08:02.994953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"56'80","reported_seq":976,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994878+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:08:02.994878+0000","last_peered":"2024-09-13T09:08:02.994878+0000","last_clean":"2024-09-13T09:08:02.994878+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:08:02.994878+0000","last_undegraded":"2024-09-13T09:08:02.994878+0000","last_fullsized":"2024-09-13T09:08:02.994878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":11264256,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":903,"num_read_kb":10673,"num_write":81,"num_write_kb":3674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944836+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:08:03.944836+0000","last_peered":"2024-09-13T09:08:03.944836+0000","last_clean":"2024-09-13T09:08:03.944836+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:08:03.944836+0000","last_undegraded":"2024-09-13T09:08:03.944836+0000","last_fullsized":"2024-09-13T09:08:03.944836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"56'256","reported_seq":675,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.174506+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:08:03.174506+0000","last_peered":"2024-09-13T09:08:03.174506+0000","last_clean":"2024-09-13T09:08:03.174506+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:08:03.174506+0000","last_undegraded":"2024-09-13T09:08:03.174506+0000","last_fullsized":"2024-09-13T09:08:03.174506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":18142525,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1343,"num_read_kb":5247,"num_write":404,"num_write_kb":1434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976644+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:08:06.976644+0000","last_peered":"2024-09-13T09:08:06.976644+0000","last_clean":"2024-09-13T09:08:06.976644+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:08:06.976644+0000","last_undegraded":"2024-09-13T09:08:06.976644+0000","last_fullsized":"2024-09-13T09:08:06.976644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"61'186","reported_seq":1954,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.014724+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:08:06.014724+0000","last_peered":"2024-09-13T09:08:06.014724+0000","last_clean":"2024-09-13T09:08:06.014724+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:08:06.014724+0000","last_undegraded":"2024-09-13T09:08:06.014724+0000","last_fullsized":"2024-09-13T09:08:06.014724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":23622460,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2392,"num_read_kb":12629,"num_write":288,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"61'109","reported_seq":1097,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709401+0000","last_change":"2024-09-13T09:07:15.873298+0000","last_active":"2024-09-13T09:08:05.709401+0000","last_peered":"2024-09-13T09:08:05.709401+0000","last_clean":"2024-09-13T09:08:05.709401+0000","last_became_active":"2024-09-13T09:07:13.852418+0000","last_became_peered":"2024-09-13T09:07:13.852418+0000","last_unstale":"2024-09-13T09:08:05.709401+0000","last_undegraded":"2024-09-13T09:08:05.709401+0000","last_fullsized":"2024-09-13T09:08:05.709401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:58.473601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":22810773,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1552,"num_read_kb":7181,"num_write":120,"num_write_kb":957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":59,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944653+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:08:03.944653+0000","last_peered":"2024-09-13T09:08:03.944653+0000","last_clean":"2024-09-13T09:08:03.944653+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:08:03.944653+0000","last_undegraded":"2024-09-13T09:08:03.944653+0000","last_fullsized":"2024-09-13T09:08:03.944653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"56'98","reported_seq":1538,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976282+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:08:06.976282+0000","last_peered":"2024-09-13T09:08:06.976282+0000","last_clean":"2024-09-13T09:08:06.976282+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:08:06.976282+0000","last_undegraded":"2024-09-13T09:08:06.976282+0000","last_fullsized":"2024-09-13T09:08:06.976282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":11014388,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":1444,"num_read_kb":10181,"num_write":100,"num_write_kb":4780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.938292+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:08:03.938292+0000","last_peered":"2024-09-13T09:08:03.938292+0000","last_clean":"2024-09-13T09:08:03.938292+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:08:03.938292+0000","last_undegraded":"2024-09-13T09:08:03.938292+0000","last_fullsized":"2024-09-13T09:08:03.938292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"61'103","reported_seq":922,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:04.218363+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:08:04.218363+0000","last_peered":"2024-09-13T09:08:04.218363+0000","last_clean":"2024-09-13T09:08:04.218363+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:08:04.218363+0000","last_undegraded":"2024-09-13T09:08:04.218363+0000","last_fullsized":"2024-09-13T09:08:04.218363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":6024682,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1019,"num_read_kb":6803,"num_write":96,"num_write_kb":3261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.943334+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:08:03.943334+0000","last_peered":"2024-09-13T09:08:03.943334+0000","last_clean":"2024-09-13T09:08:03.943334+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:08:03.943334+0000","last_undegraded":"2024-09-13T09:08:03.943334+0000","last_fullsized":"2024-09-13T09:08:03.943334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"56'130","reported_seq":309,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994707+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:08:02.994707+0000","last_peered":"2024-09-13T09:08:02.994707+0000","last_clean":"2024-09-13T09:08:02.994707+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:08:02.994707+0000","last_undegraded":"2024-09-13T09:08:02.994707+0000","last_fullsized":"2024-09-13T09:08:02.994707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":1465988,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":473,"num_read_kb":3329,"num_write":175,"num_write_kb":1460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":58,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944810+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:08:03.944810+0000","last_peered":"2024-09-13T09:08:03.944810+0000","last_clean":"2024-09-13T09:08:03.944810+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:08:03.944810+0000","last_undegraded":"2024-09-13T09:08:03.944810+0000","last_fullsized":"2024-09-13T09:08:03.944810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:04.218502+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:08:04.218502+0000","last_peered":"2024-09-13T09:08:04.218502+0000","last_clean":"2024-09-13T09:08:04.218502+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:08:04.218502+0000","last_undegraded":"2024-09-13T09:08:04.218502+0000","last_fullsized":"2024-09-13T09:08:04.218502+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"57'153","reported_seq":1984,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.178847+0000","last_change":"2024-09-13T09:07:16.108021+0000","last_active":"2024-09-13T09:08:03.178847+0000","last_peered":"2024-09-13T09:08:03.178847+0000","last_clean":"2024-09-13T09:08:03.178847+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:08:03.178847+0000","last_undegraded":"2024-09-13T09:08:03.178847+0000","last_fullsized":"2024-09-13T09:08:03.178847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":26026737,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2139,"num_read_kb":12139,"num_write":296,"num_write_kb":7443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"61'220","reported_seq":2078,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.705592+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:08:05.705592+0000","last_peered":"2024-09-13T09:08:05.705592+0000","last_clean":"2024-09-13T09:08:05.705592+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:08:05.705592+0000","last_undegraded":"2024-09-13T09:08:05.705592+0000","last_fullsized":"2024-09-13T09:08:05.705592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":33348128,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":2951,"num_read_kb":10450,"num_write":297,"num_write_kb":4571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"61'150","reported_seq":704,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.706520+0000","last_change":"2024-09-13T09:07:53.864337+0000","last_active":"2024-09-13T09:08:05.706520+0000","last_peered":"2024-09-13T09:08:05.706520+0000","last_clean":"2024-09-13T09:08:05.706520+0000","last_became_active":"2024-09-13T09:07:53.862992+0000","last_became_peered":"2024-09-13T09:07:53.862992+0000","last_unstale":"2024-09-13T09:08:05.706520+0000","last_undegraded":"2024-09-13T09:08:05.706520+0000","last_fullsized":"2024-09-13T09:08:05.706520+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:14.717209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":33289951,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":480,"num_read_kb":5891,"num_write":155,"num_write_kb":7563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":2632291,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"56'70","reported_seq":1269,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.938382+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:08:03.938382+0000","last_peered":"2024-09-13T09:08:03.938382+0000","last_clean":"2024-09-13T09:08:03.938382+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:08:03.938382+0000","last_undegraded":"2024-09-13T09:08:03.938382+0000","last_fullsized":"2024-09-13T09:08:03.938382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":8722868,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1189,"num_read_kb":13116,"num_write":94,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"61'90","reported_seq":15663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.848616+0000","last_change":"2024-09-13T09:07:15.873583+0000","last_active":"2024-09-13T09:08:05.848616+0000","last_peered":"2024-09-13T09:08:05.848616+0000","last_clean":"2024-09-13T09:08:05.848616+0000","last_became_active":"2024-09-13T09:07:13.859769+0000","last_became_peered":"2024-09-13T09:07:13.859769+0000","last_unstale":"2024-09-13T09:08:05.848616+0000","last_undegraded":"2024-09-13T09:08:05.848616+0000","last_fullsized":"2024-09-13T09:08:05.848616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:52.161129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":40221705,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":15551,"num_read_kb":23982,"num_write":139,"num_write_kb":12633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"61'63","reported_seq":1216,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709177+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:08:05.709177+0000","last_peered":"2024-09-13T09:08:05.709177+0000","last_clean":"2024-09-13T09:08:05.709177+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:08:05.709177+0000","last_undegraded":"2024-09-13T09:08:05.709177+0000","last_fullsized":"2024-09-13T09:08:05.709177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":9877649,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1029,"num_read_kb":5678,"num_write":63,"num_write_kb":1398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"61'72","reported_seq":2147,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709780+0000","last_change":"2024-09-13T09:07:15.859429+0000","last_active":"2024-09-13T09:08:05.709780+0000","last_peered":"2024-09-13T09:08:05.709780+0000","last_clean":"2024-09-13T09:08:05.709780+0000","last_became_active":"2024-09-13T09:07:13.855153+0000","last_became_peered":"2024-09-13T09:07:13.855153+0000","last_unstale":"2024-09-13T09:08:05.709780+0000","last_undegraded":"2024-09-13T09:08:05.709780+0000","last_fullsized":"2024-09-13T09:08:05.709780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:15.261153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":21285344,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":2044,"num_read_kb":9681,"num_write":69,"num_write_kb":1724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"56'120","reported_seq":1346,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.974106+0000","last_change":"2024-09-13T09:07:51.052606+0000","last_active":"2024-09-13T09:08:03.974106+0000","last_peered":"2024-09-13T09:08:03.974106+0000","last_clean":"2024-09-13T09:08:03.974106+0000","last_became_active":"2024-09-13T09:07:50.848333+0000","last_became_peered":"2024-09-13T09:07:50.848333+0000","last_unstale":"2024-09-13T09:08:03.974106+0000","last_undegraded":"2024-09-13T09:08:03.974106+0000","last_fullsized":"2024-09-13T09:08:03.974106+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:23:05.120217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":12634663,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":1522,"num_read_kb":4341,"num_write":119,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":885530,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"61'163","reported_seq":1546,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-13T09:08:05.963741+0000","last_change":"2024-09-13T09:08:05.964022+0000","last_active":"2024-09-13T09:08:05.963741+0000","last_peered":"2024-09-13T09:08:05.893968+0000","last_clean":"2024-09-13T09:08:05.893968+0000","last_became_active":"2024-09-13T09:07:53.861512+0000","last_became_peered":"2024-09-13T09:07:53.861512+0000","last_unstale":"2024-09-13T09:08:05.963741+0000","last_undegraded":"2024-09-13T09:08:05.963741+0000","last_fullsized":"2024-09-13T09:08:05.963741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:46:53.676923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":46116942,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":1444,"num_read_kb":13900,"num_write":179,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":6431007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"62'423","reported_seq":1645,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.028591+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:08:06.028591+0000","last_peered":"2024-09-13T09:08:06.028591+0000","last_clean":"2024-09-13T09:08:06.028591+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:08:06.028591+0000","last_undegraded":"2024-09-13T09:08:06.028591+0000","last_fullsized":"2024-09-13T09:08:06.028591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":15305898,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":3517,"num_read_kb":4793,"num_write":694,"num_write_kb":3399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"56'192","reported_seq":1933,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.930510+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:08:02.930510+0000","last_peered":"2024-09-13T09:08:02.930510+0000","last_clean":"2024-09-13T09:08:02.930510+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:08:02.930510+0000","last_undegraded":"2024-09-13T09:08:02.930510+0000","last_fullsized":"2024-09-13T09:08:02.930510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":27927802,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2773,"num_read_kb":7288,"num_write":288,"num_write_kb":4024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"56'190","reported_seq":909,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944685+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:08:03.944685+0000","last_peered":"2024-09-13T09:08:03.944685+0000","last_clean":"2024-09-13T09:08:03.944685+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:08:03.944685+0000","last_undegraded":"2024-09-13T09:08:03.944685+0000","last_fullsized":"2024-09-13T09:08:03.944685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":4364767,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":1626,"num_read_kb":5582,"num_write":271,"num_write_kb":4745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":773464315,"num_objects":629,"num_object_clones":160,"num_object_copies":1258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":169,"num_read":104535,"num_read_kb":363418,"num_write":6987,"num_write_kb":201048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":47163705,"num_keys_recovered":68,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":254537728,"data_stored":267497534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5388,"ondisk_log_size":5388,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60208,"kb_used_data":24996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94311632,"statfs":{"total":96636764160,"available":96575111168,"internally_reserved":0,"allocated":25595904,"data_stored":25969500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.88700000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.194},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.169}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.286}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0389999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72948,"kb_used_data":37740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94298892,"statfs":{"total":96636764160,"available":96562065408,"internally_reserved":0,"allocated":38645760,"data_stored":41642616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.377}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.44400000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.624}]}]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52452,"kb_used_data":17224,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94319388,"statfs":{"total":96636764160,"available":96583053312,"internally_reserved":0,"allocated":17637376,"data_stored":17831540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.67300000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60632,"kb_used_data":29520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94311208,"statfs":{"total":96636764160,"available":96574676992,"internally_reserved":0,"allocated":30228480,"data_stored":29638714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":2.2650000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.309}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.874},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.57699999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75200,"kb_used_data":44084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94296640,"statfs":{"total":96636764160,"available":96559759360,"internally_reserved":0,"allocated":45142016,"data_stored":50164914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.37}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.80900000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55760,"kb_used_data":20540,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94316080,"statfs":{"total":96636764160,"available":96579665920,"internally_reserved":0,"allocated":21032960,"data_stored":24108649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.79000000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.89700000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.871},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1459999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73184,"kb_used_data":33868,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94298656,"statfs":{"total":96636764160,"available":96561823744,"internally_reserved":0,"allocated":34680832,"data_stored":34068846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.501},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.748}]},{"osd":2,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.622},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.97999999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82799999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.85199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76444,"kb_used_data":37140,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94295396,"statfs":{"total":96636764160,"available":96558485504,"internally_reserved":0,"allocated":38031360,"data_stored":37391851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.501},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.9740000000000002,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.9050000000000002,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":8.0850000000000009,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44429312,"data_stored":49848197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31068160,"data_stored":30865820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19062784,"data_stored":19645440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41340928,"data_stored":44720649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33980416,"data_stored":33757952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":39424000,"data_stored":39189561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20348928,"data_stored":23817132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24883200,"data_stored":25652783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:08:09.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:08:09.388 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:09.385+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.4472401930000001, 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.475 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.4471928599999999, 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:08:09.480 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:08:09.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:08:09.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.447135324, 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:08:09.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:08:09.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.489 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.447083838, 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:08:09.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.494 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.447036896, 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:08:09.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:08:09.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:09.499 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.446989503, 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:09.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:08:09.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.4469422279999999, 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:08:09.504 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:08:09.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.4468960280000001, 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.509 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:09.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.513 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.446832023, 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:08:09.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:08:09.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:09.518 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.4443609340000001, 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:09.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:08:09.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.4443052119999999, 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.525 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:09.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.529 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.4442553089999999, 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.530 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:08:09.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.444208253, 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.535 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:08:09.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.44415977, 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.540 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:08:09.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.544 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.444094505, 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:08:09.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:08:09.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.444047589, 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.549 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.550 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:08:09.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.553 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.443960141, 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:08:09.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.558 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.442498292, 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:09.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:08:09.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.442420206, 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.563 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.564 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.565 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.566 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:08:09.567 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.442368117, 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:09.568 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:09.569 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.570 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:08:09.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.572 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.573 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:09.573 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:09.573 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:09.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:09.575 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:09.567+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:09.675 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:09.669+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:09.700 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:09.701 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:08:09.719 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-13T09:08:09.064978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":774054702,"num_objects":633,"num_object_clones":160,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":169,"num_read":104651,"num_read_kb":363518,"num_write":7148,"num_write_kb":203466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":47754073,"num_keys_recovered":68,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5508,"ondisk_log_size":5508,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":526828,"kb_used_data":245112,"kb_used_omap":12,"kb_used_meta":281587,"kb_avail":754447892,"statfs":{"total":773094113280,"available":772554641408,"internally_reserved":0,"allocated":250994688,"data_stored":260816630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":288345656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":100419972,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":30017,"num_read_kb":35394,"num_write":247,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001245"},"pg_stats":[{"pgid":"3.1f","version":"64'160","reported_seq":668,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.171013+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:08:08.171013+0000","last_peered":"2024-09-13T09:08:08.171013+0000","last_clean":"2024-09-13T09:08:08.171013+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:08:08.171013+0000","last_undegraded":"2024-09-13T09:08:08.171013+0000","last_fullsized":"2024-09-13T09:08:08.171013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":10845646,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":843,"num_read_kb":4918,"num_write":213,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"64'208","reported_seq":1695,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.209774+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:08:08.209774+0000","last_peered":"2024-09-13T09:08:08.209774+0000","last_clean":"2024-09-13T09:08:08.209774+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:08:08.209774+0000","last_undegraded":"2024-09-13T09:08:08.209774+0000","last_fullsized":"2024-09-13T09:08:08.209774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":37602897,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":2432,"num_read_kb":9910,"num_write":243,"num_write_kb":3848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"60'179","reported_seq":2312,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.669984+0000","last_change":"2024-09-13T09:07:15.879980+0000","last_active":"2024-09-13T09:08:05.669984+0000","last_peered":"2024-09-13T09:08:05.669984+0000","last_clean":"2024-09-13T09:08:05.669984+0000","last_became_active":"2024-09-13T09:07:13.856004+0000","last_became_peered":"2024-09-13T09:07:13.856004+0000","last_unstale":"2024-09-13T09:08:05.669984+0000","last_undegraded":"2024-09-13T09:08:05.669984+0000","last_fullsized":"2024-09-13T09:08:05.669984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:37.046343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":48416107,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":2124,"num_read_kb":25278,"num_write":253,"num_write_kb":13248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"61'85","reported_seq":1507,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.004737+0000","last_change":"2024-09-13T09:07:51.093764+0000","last_active":"2024-09-13T09:08:06.004737+0000","last_peered":"2024-09-13T09:08:06.004737+0000","last_clean":"2024-09-13T09:08:06.004737+0000","last_became_active":"2024-09-13T09:07:50.836531+0000","last_became_peered":"2024-09-13T09:07:50.836531+0000","last_unstale":"2024-09-13T09:08:06.004737+0000","last_undegraded":"2024-09-13T09:08:06.004737+0000","last_fullsized":"2024-09-13T09:08:06.004737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:11.585290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":46102973,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2515,"num_read_kb":22664,"num_write":119,"num_write_kb":10107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":3944384,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"60'268","reported_seq":17538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.434612+0000","last_change":"2024-09-13T09:07:16.108727+0000","last_active":"2024-09-13T09:08:05.434612+0000","last_peered":"2024-09-13T09:08:05.434612+0000","last_clean":"2024-09-13T09:08:05.434612+0000","last_became_active":"2024-09-13T09:07:13.862565+0000","last_became_peered":"2024-09-13T09:07:13.862565+0000","last_unstale":"2024-09-13T09:08:05.434612+0000","last_undegraded":"2024-09-13T09:08:05.434612+0000","last_fullsized":"2024-09-13T09:08:05.434612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:32.305423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":58876863,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":18533,"num_read_kb":25518,"num_write":341,"num_write_kb":16770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"61'180","reported_seq":1945,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-13T09:08:05.963785+0000","last_change":"2024-09-13T09:08:05.964026+0000","last_active":"2024-09-13T09:08:05.963785+0000","last_peered":"2024-09-13T09:08:05.894139+0000","last_clean":"2024-09-13T09:08:05.894139+0000","last_became_active":"2024-09-13T09:07:50.833042+0000","last_became_peered":"2024-09-13T09:07:50.833042+0000","last_unstale":"2024-09-13T09:08:05.963785+0000","last_undegraded":"2024-09-13T09:08:05.963785+0000","last_fullsized":"2024-09-13T09:08:05.963785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:47.212272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":47325778,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":3012,"num_read_kb":13732,"num_write":314,"num_write_kb":16562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":31801546,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"60'93","reported_seq":749,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.175265+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:08:03.175265+0000","last_peered":"2024-09-13T09:08:03.175265+0000","last_clean":"2024-09-13T09:08:03.175265+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:08:03.175265+0000","last_undegraded":"2024-09-13T09:08:03.175265+0000","last_fullsized":"2024-09-13T09:08:03.175265+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":2509931,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":580,"num_read_kb":3310,"num_write":93,"num_write_kb":4753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"59'154","reported_seq":1220,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976447+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:08:06.976447+0000","last_peered":"2024-09-13T09:08:06.976447+0000","last_clean":"2024-09-13T09:08:06.976447+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:08:06.976447+0000","last_undegraded":"2024-09-13T09:08:06.976447+0000","last_fullsized":"2024-09-13T09:08:06.976447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":18577308,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":1714,"num_read_kb":8005,"num_write":153,"num_write_kb":3690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"62'240","reported_seq":2822,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976621+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:08:06.976621+0000","last_peered":"2024-09-13T09:08:06.976621+0000","last_clean":"2024-09-13T09:08:06.976621+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:08:06.976621+0000","last_undegraded":"2024-09-13T09:08:06.976621+0000","last_fullsized":"2024-09-13T09:08:06.976621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":42969160,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3400,"num_read_kb":21858,"num_write":270,"num_write_kb":16106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"61'110","reported_seq":625,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.848566+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:08:05.848566+0000","last_peered":"2024-09-13T09:08:05.848566+0000","last_clean":"2024-09-13T09:08:05.848566+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:08:05.848566+0000","last_undegraded":"2024-09-13T09:08:05.848566+0000","last_fullsized":"2024-09-13T09:08:05.848566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":9332974,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":639,"num_read_kb":3988,"num_write":126,"num_write_kb":3362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"61'205","reported_seq":2005,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.708645+0000","last_change":"2024-09-13T09:07:15.873331+0000","last_active":"2024-09-13T09:08:05.708645+0000","last_peered":"2024-09-13T09:08:05.708645+0000","last_clean":"2024-09-13T09:08:05.708645+0000","last_became_active":"2024-09-13T09:07:13.866813+0000","last_became_peered":"2024-09-13T09:07:13.866813+0000","last_unstale":"2024-09-13T09:08:05.708645+0000","last_undegraded":"2024-09-13T09:08:05.708645+0000","last_fullsized":"2024-09-13T09:08:05.708645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.439602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":45587035,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":2341,"num_read_kb":17771,"num_write":219,"num_write_kb":11683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"61'286","reported_seq":17229,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.058722+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:08:05.058722+0000","last_peered":"2024-09-13T09:08:05.058722+0000","last_clean":"2024-09-13T09:08:05.058722+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:08:05.058722+0000","last_undegraded":"2024-09-13T09:08:05.058722+0000","last_fullsized":"2024-09-13T09:08:05.058722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":16839736,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":5,"num_read":18600,"num_read_kb":25170,"num_write":260,"num_write_kb":6627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944805+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:08:03.944805+0000","last_peered":"2024-09-13T09:08:03.944805+0000","last_clean":"2024-09-13T09:08:03.944805+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:08:03.944805+0000","last_undegraded":"2024-09-13T09:08:03.944805+0000","last_fullsized":"2024-09-13T09:08:03.944805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"56'352","reported_seq":1373,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.178985+0000","last_change":"2024-09-13T09:07:16.108420+0000","last_active":"2024-09-13T09:08:03.178985+0000","last_peered":"2024-09-13T09:08:03.178985+0000","last_clean":"2024-09-13T09:08:03.178985+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:08:03.178985+0000","last_undegraded":"2024-09-13T09:08:03.178985+0000","last_fullsized":"2024-09-13T09:08:03.178985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":15010381,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":2411,"num_read_kb":8412,"num_write":455,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994953+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:08:02.994953+0000","last_peered":"2024-09-13T09:08:02.994953+0000","last_clean":"2024-09-13T09:08:02.994953+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:08:02.994953+0000","last_undegraded":"2024-09-13T09:08:02.994953+0000","last_fullsized":"2024-09-13T09:08:02.994953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"56'80","reported_seq":976,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994878+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:08:02.994878+0000","last_peered":"2024-09-13T09:08:02.994878+0000","last_clean":"2024-09-13T09:08:02.994878+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:08:02.994878+0000","last_undegraded":"2024-09-13T09:08:02.994878+0000","last_fullsized":"2024-09-13T09:08:02.994878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":11264256,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":903,"num_read_kb":10673,"num_write":81,"num_write_kb":3674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944836+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:08:03.944836+0000","last_peered":"2024-09-13T09:08:03.944836+0000","last_clean":"2024-09-13T09:08:03.944836+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:08:03.944836+0000","last_undegraded":"2024-09-13T09:08:03.944836+0000","last_fullsized":"2024-09-13T09:08:03.944836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"56'256","reported_seq":675,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.174506+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:08:03.174506+0000","last_peered":"2024-09-13T09:08:03.174506+0000","last_clean":"2024-09-13T09:08:03.174506+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:08:03.174506+0000","last_undegraded":"2024-09-13T09:08:03.174506+0000","last_fullsized":"2024-09-13T09:08:03.174506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":18142525,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1343,"num_read_kb":5247,"num_write":404,"num_write_kb":1434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976644+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:08:06.976644+0000","last_peered":"2024-09-13T09:08:06.976644+0000","last_clean":"2024-09-13T09:08:06.976644+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:08:06.976644+0000","last_undegraded":"2024-09-13T09:08:06.976644+0000","last_fullsized":"2024-09-13T09:08:06.976644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"61'186","reported_seq":1954,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.014724+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:08:06.014724+0000","last_peered":"2024-09-13T09:08:06.014724+0000","last_clean":"2024-09-13T09:08:06.014724+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:08:06.014724+0000","last_undegraded":"2024-09-13T09:08:06.014724+0000","last_fullsized":"2024-09-13T09:08:06.014724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":23622460,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2392,"num_read_kb":12629,"num_write":288,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"61'109","reported_seq":1097,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709401+0000","last_change":"2024-09-13T09:07:15.873298+0000","last_active":"2024-09-13T09:08:05.709401+0000","last_peered":"2024-09-13T09:08:05.709401+0000","last_clean":"2024-09-13T09:08:05.709401+0000","last_became_active":"2024-09-13T09:07:13.852418+0000","last_became_peered":"2024-09-13T09:07:13.852418+0000","last_unstale":"2024-09-13T09:08:05.709401+0000","last_undegraded":"2024-09-13T09:08:05.709401+0000","last_fullsized":"2024-09-13T09:08:05.709401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:58.473601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":22810773,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1552,"num_read_kb":7181,"num_write":120,"num_write_kb":957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":59,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944653+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:08:03.944653+0000","last_peered":"2024-09-13T09:08:03.944653+0000","last_clean":"2024-09-13T09:08:03.944653+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:08:03.944653+0000","last_undegraded":"2024-09-13T09:08:03.944653+0000","last_fullsized":"2024-09-13T09:08:03.944653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"56'98","reported_seq":1538,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976282+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:08:06.976282+0000","last_peered":"2024-09-13T09:08:06.976282+0000","last_clean":"2024-09-13T09:08:06.976282+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:08:06.976282+0000","last_undegraded":"2024-09-13T09:08:06.976282+0000","last_fullsized":"2024-09-13T09:08:06.976282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":11014388,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":1444,"num_read_kb":10181,"num_write":100,"num_write_kb":4780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.938292+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:08:03.938292+0000","last_peered":"2024-09-13T09:08:03.938292+0000","last_clean":"2024-09-13T09:08:03.938292+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:08:03.938292+0000","last_undegraded":"2024-09-13T09:08:03.938292+0000","last_fullsized":"2024-09-13T09:08:03.938292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"61'103","reported_seq":922,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:04.218363+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:08:04.218363+0000","last_peered":"2024-09-13T09:08:04.218363+0000","last_clean":"2024-09-13T09:08:04.218363+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:08:04.218363+0000","last_undegraded":"2024-09-13T09:08:04.218363+0000","last_fullsized":"2024-09-13T09:08:04.218363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":6024682,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1019,"num_read_kb":6803,"num_write":96,"num_write_kb":3261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.943334+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:08:03.943334+0000","last_peered":"2024-09-13T09:08:03.943334+0000","last_clean":"2024-09-13T09:08:03.943334+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:08:03.943334+0000","last_undegraded":"2024-09-13T09:08:03.943334+0000","last_fullsized":"2024-09-13T09:08:03.943334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"56'130","reported_seq":309,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994707+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:08:02.994707+0000","last_peered":"2024-09-13T09:08:02.994707+0000","last_clean":"2024-09-13T09:08:02.994707+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:08:02.994707+0000","last_undegraded":"2024-09-13T09:08:02.994707+0000","last_fullsized":"2024-09-13T09:08:02.994707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":1465988,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":473,"num_read_kb":3329,"num_write":175,"num_write_kb":1460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":58,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944810+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:08:03.944810+0000","last_peered":"2024-09-13T09:08:03.944810+0000","last_clean":"2024-09-13T09:08:03.944810+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:08:03.944810+0000","last_undegraded":"2024-09-13T09:08:03.944810+0000","last_fullsized":"2024-09-13T09:08:03.944810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:04.218502+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:08:04.218502+0000","last_peered":"2024-09-13T09:08:04.218502+0000","last_clean":"2024-09-13T09:08:04.218502+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:08:04.218502+0000","last_undegraded":"2024-09-13T09:08:04.218502+0000","last_fullsized":"2024-09-13T09:08:04.218502+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"57'153","reported_seq":1984,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.178847+0000","last_change":"2024-09-13T09:07:16.108021+0000","last_active":"2024-09-13T09:08:03.178847+0000","last_peered":"2024-09-13T09:08:03.178847+0000","last_clean":"2024-09-13T09:08:03.178847+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:08:03.178847+0000","last_undegraded":"2024-09-13T09:08:03.178847+0000","last_fullsized":"2024-09-13T09:08:03.178847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":26026737,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2139,"num_read_kb":12139,"num_write":296,"num_write_kb":7443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"61'220","reported_seq":2078,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.705592+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:08:05.705592+0000","last_peered":"2024-09-13T09:08:05.705592+0000","last_clean":"2024-09-13T09:08:05.705592+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:08:05.705592+0000","last_undegraded":"2024-09-13T09:08:05.705592+0000","last_fullsized":"2024-09-13T09:08:05.705592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":33348128,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":2951,"num_read_kb":10450,"num_write":297,"num_write_kb":4571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"61'150","reported_seq":704,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.706520+0000","last_change":"2024-09-13T09:07:53.864337+0000","last_active":"2024-09-13T09:08:05.706520+0000","last_peered":"2024-09-13T09:08:05.706520+0000","last_clean":"2024-09-13T09:08:05.706520+0000","last_became_active":"2024-09-13T09:07:53.862992+0000","last_became_peered":"2024-09-13T09:07:53.862992+0000","last_unstale":"2024-09-13T09:08:05.706520+0000","last_undegraded":"2024-09-13T09:08:05.706520+0000","last_fullsized":"2024-09-13T09:08:05.706520+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:14.717209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":33289951,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":480,"num_read_kb":5891,"num_write":155,"num_write_kb":7563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":2632291,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"56'70","reported_seq":1269,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.938382+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:08:03.938382+0000","last_peered":"2024-09-13T09:08:03.938382+0000","last_clean":"2024-09-13T09:08:03.938382+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:08:03.938382+0000","last_undegraded":"2024-09-13T09:08:03.938382+0000","last_fullsized":"2024-09-13T09:08:03.938382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":8722868,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1189,"num_read_kb":13116,"num_write":94,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"61'90","reported_seq":15663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.848616+0000","last_change":"2024-09-13T09:07:15.873583+0000","last_active":"2024-09-13T09:08:05.848616+0000","last_peered":"2024-09-13T09:08:05.848616+0000","last_clean":"2024-09-13T09:08:05.848616+0000","last_became_active":"2024-09-13T09:07:13.859769+0000","last_became_peered":"2024-09-13T09:07:13.859769+0000","last_unstale":"2024-09-13T09:08:05.848616+0000","last_undegraded":"2024-09-13T09:08:05.848616+0000","last_fullsized":"2024-09-13T09:08:05.848616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:52.161129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":40221705,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":15551,"num_read_kb":23982,"num_write":139,"num_write_kb":12633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"61'63","reported_seq":1216,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709177+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:08:05.709177+0000","last_peered":"2024-09-13T09:08:05.709177+0000","last_clean":"2024-09-13T09:08:05.709177+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:08:05.709177+0000","last_undegraded":"2024-09-13T09:08:05.709177+0000","last_fullsized":"2024-09-13T09:08:05.709177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":9877649,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1029,"num_read_kb":5678,"num_write":63,"num_write_kb":1398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"61'72","reported_seq":2147,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709780+0000","last_change":"2024-09-13T09:07:15.859429+0000","last_active":"2024-09-13T09:08:05.709780+0000","last_peered":"2024-09-13T09:08:05.709780+0000","last_clean":"2024-09-13T09:08:05.709780+0000","last_became_active":"2024-09-13T09:07:13.855153+0000","last_became_peered":"2024-09-13T09:07:13.855153+0000","last_unstale":"2024-09-13T09:08:05.709780+0000","last_undegraded":"2024-09-13T09:08:05.709780+0000","last_fullsized":"2024-09-13T09:08:05.709780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:15.261153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":21285344,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":2044,"num_read_kb":9681,"num_write":69,"num_write_kb":1724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"56'120","reported_seq":1346,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.974106+0000","last_change":"2024-09-13T09:07:51.052606+0000","last_active":"2024-09-13T09:08:03.974106+0000","last_peered":"2024-09-13T09:08:03.974106+0000","last_clean":"2024-09-13T09:08:03.974106+0000","last_became_active":"2024-09-13T09:07:50.848333+0000","last_became_peered":"2024-09-13T09:07:50.848333+0000","last_unstale":"2024-09-13T09:08:03.974106+0000","last_undegraded":"2024-09-13T09:08:03.974106+0000","last_fullsized":"2024-09-13T09:08:03.974106+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:23:05.120217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":12634663,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":1522,"num_read_kb":4341,"num_write":119,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":885530,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"61'163","reported_seq":1546,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-13T09:08:05.963741+0000","last_change":"2024-09-13T09:08:05.964022+0000","last_active":"2024-09-13T09:08:05.963741+0000","last_peered":"2024-09-13T09:08:05.893968+0000","last_clean":"2024-09-13T09:08:05.893968+0000","last_became_active":"2024-09-13T09:07:53.861512+0000","last_became_peered":"2024-09-13T09:07:53.861512+0000","last_unstale":"2024-09-13T09:08:05.963741+0000","last_undegraded":"2024-09-13T09:08:05.963741+0000","last_fullsized":"2024-09-13T09:08:05.963741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:46:53.676923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":46116942,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":1444,"num_read_kb":13900,"num_write":179,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":6431007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"62'423","reported_seq":1645,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.028591+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:08:06.028591+0000","last_peered":"2024-09-13T09:08:06.028591+0000","last_clean":"2024-09-13T09:08:06.028591+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:08:06.028591+0000","last_undegraded":"2024-09-13T09:08:06.028591+0000","last_fullsized":"2024-09-13T09:08:06.028591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":15305898,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":3517,"num_read_kb":4793,"num_write":694,"num_write_kb":3399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"56'192","reported_seq":1933,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.930510+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:08:02.930510+0000","last_peered":"2024-09-13T09:08:02.930510+0000","last_clean":"2024-09-13T09:08:02.930510+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:08:02.930510+0000","last_undegraded":"2024-09-13T09:08:02.930510+0000","last_fullsized":"2024-09-13T09:08:02.930510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":27927802,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2773,"num_read_kb":7288,"num_write":288,"num_write_kb":4024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"56'190","reported_seq":909,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944685+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:08:03.944685+0000","last_peered":"2024-09-13T09:08:03.944685+0000","last_clean":"2024-09-13T09:08:03.944685+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:08:03.944685+0000","last_undegraded":"2024-09-13T09:08:03.944685+0000","last_fullsized":"2024-09-13T09:08:03.944685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":4364767,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":1626,"num_read_kb":5582,"num_write":271,"num_write_kb":4745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":773464315,"num_objects":629,"num_object_clones":160,"num_object_copies":1258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":169,"num_read":104535,"num_read_kb":363418,"num_write":6987,"num_write_kb":201048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":47163705,"num_keys_recovered":68,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":254537728,"data_stored":267497534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5388,"ondisk_log_size":5388,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60208,"kb_used_data":24996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94311632,"statfs":{"total":96636764160,"available":96575111168,"internally_reserved":0,"allocated":25595904,"data_stored":25969500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.88700000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.194},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.169}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.286}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0389999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72948,"kb_used_data":37740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94298892,"statfs":{"total":96636764160,"available":96562065408,"internally_reserved":0,"allocated":38645760,"data_stored":41642616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.377}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.44400000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.624}]}]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52452,"kb_used_data":17224,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94319388,"statfs":{"total":96636764160,"available":96583053312,"internally_reserved":0,"allocated":17637376,"data_stored":17831540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.67300000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60632,"kb_used_data":29520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94311208,"statfs":{"total":96636764160,"available":96574676992,"internally_reserved":0,"allocated":30228480,"data_stored":29638714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":2.2650000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.309}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.874},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.57699999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75200,"kb_used_data":44084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94296640,"statfs":{"total":96636764160,"available":96559759360,"internally_reserved":0,"allocated":45142016,"data_stored":50164914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.37}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.80900000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55760,"kb_used_data":20540,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94316080,"statfs":{"total":96636764160,"available":96579665920,"internally_reserved":0,"allocated":21032960,"data_stored":24108649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.79000000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.89700000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.871},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1459999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73184,"kb_used_data":33868,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94298656,"statfs":{"total":96636764160,"available":96561823744,"internally_reserved":0,"allocated":34680832,"data_stored":34068846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.501},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.748}]},{"osd":2,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.622},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.97999999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82799999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.85199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76444,"kb_used_data":37140,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94295396,"statfs":{"total":96636764160,"available":96558485504,"internally_reserved":0,"allocated":38031360,"data_stored":37391851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.501},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.9740000000000002,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.9050000000000002,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":8.0850000000000009,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44429312,"data_stored":49848197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31068160,"data_stored":30865820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19062784,"data_stored":19645440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41340928,"data_stored":44720649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33980416,"data_stored":33757952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":39424000,"data_stored":39189561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20348928,"data_stored":23817132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24883200,"data_stored":25652783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:08:09.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T09:08:09.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:09.749 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:09.749 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:09.750 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:09.750 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:09.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:09.807 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:09.803+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:09.858 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:08:09.876 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:09.873+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:09.915 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:09.915 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:09.931 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:09.944 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:09.945 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:09.945 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:09.945 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:09.945 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:09.947 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:09.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:09.962 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone19 intact 2024-09-13T09:08:09.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:09.965 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:09.982 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:09.983 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:10.002 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:10.002 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:10.009 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:10.005+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:10.022 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:10.023 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:10.050 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:10.078 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:10.073+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-13T09:08:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:2171 punch from 0x2a0e687 to 0x2a10930, (0x22a9 bytes) 2024-09-13T09:08:10.141 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:10.141 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:2173 write 0xcfe7465 thru 0xcff1d03 (0xa89f bytes) 2024-09-13T09:08:10.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:10.174 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":86,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":66,"num_osds":8,"num_up_osds":8,"osd_up_since":1726218415,"num_in_osds":8,"osd_in_since":1726218469,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":39},{"state_name":"peering","count":2}],"num_pgs":41,"num_pools":3,"num_objects":633,"data_bytes":774054702,"bytes_used":539471872,"bytes_avail":772554641408,"bytes_total":773094113280,"inactive_pgs_ratio":0.048780485987663269,"read_bytes_sec":9058043,"write_bytes_sec":6464547,"read_op_per_sec":7501,"write_op_per_sec":61},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-13T09:07:57.061453+0000","services":{}},"progress_events":{}} 2024-09-13T09:08:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:2174 writesame 0x2052361 thru 0x205ec12 data_size 0x3(0xc8b2 bytes) 2024-09-13T09:08:10.179 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:10.175+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:10.198 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-13T09:08:10.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:08:10.199 INFO:teuthology.orchestra.run.smithi031.stdout:2175 writesame 0x5b0d267 thru 0x5b1277a data_size 0x3c(0x5514 bytes) 2024-09-13T09:08:10.217 INFO:teuthology.orchestra.run.smithi031.stdout:2176 read 0x8a92669 thru 0x8a9d2e5 (0xac7d bytes) 2024-09-13T09:08:10.218 INFO:teuthology.orchestra.run.smithi031.stdout:2178 writesame 0x2f5c002 thru 0x2f5f418 data_size 0x23(0x3417 bytes) 2024-09-13T09:08:10.229 INFO:teuthology.orchestra.run.smithi031.stdout:2179 write 0x299ad53 thru 0x29a6d57 (0xc005 bytes) 2024-09-13T09:08:10.242 INFO:teuthology.orchestra.run.smithi031.stdout:2180 read 0x4c7b562 thru 0x4c85db1 (0xa850 bytes) 2024-09-13T09:08:10.242 INFO:teuthology.orchestra.run.smithi031.stdout:2181 write 0x26e082c thru 0x26e4f86 (0x475b bytes) 2024-09-13T09:08:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:2183 write 0x5b112ff thru 0x5b1c4ba (0xb1bc bytes) 2024-09-13T09:08:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:2185 trunc from 0xcff1d04 to 0x1b7a680 2024-09-13T09:08:10.280 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:10.275+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:10.360 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:10.360 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:10.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:2186 write 0x8700b8a thru 0x870702d (0x64a4 bytes) 2024-09-13T09:08:10.419 INFO:teuthology.orchestra.run.smithi031.stdout:2187 write 0x6decaf7 thru 0x6df963e (0xcb48 bytes) 2024-09-13T09:08:10.428 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:10.425+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:10.433 INFO:teuthology.orchestra.run.smithi031.stdout:2188 trunc from 0x870702e to 0x9e1a0be 2024-09-13T09:08:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:2189 read 0x2f8fb01 thru 0x2f91c79 (0x2179 bytes) 2024-09-13T09:08:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:2190 write 0x5b3023c thru 0x5b3df35 (0xdcfa bytes) 2024-09-13T09:08:10.459 INFO:teuthology.orchestra.run.smithi031.stdout:2192 write 0xabf7623 thru 0xabf8a5c (0x143a bytes) 2024-09-13T09:08:10.470 INFO:teuthology.orchestra.run.smithi031.stdout:2193 trunc from 0xabf8a5d to 0xeacdc3d 2024-09-13T09:08:10.476 INFO:teuthology.orchestra.run.smithi031.stdout:2195 punch from 0x572d85a to 0x573a884, (0xd02a bytes) 2024-09-13T09:08:10.482 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:10.477+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:10.490 INFO:teuthology.orchestra.run.smithi031.stdout:2197 trunc from 0xeacdc3d to 0x22c499c 2024-09-13T09:08:10.507 INFO:teuthology.orchestra.run.smithi031.stdout:2198 clone 21 order 21 su 262144 sc 4 2024-09-13T09:08:10.512 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:10.512 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:10.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:10.560 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:08:10.576 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-13T09:08:09.852225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":774054702,"num_objects":633,"num_object_clones":160,"num_object_copies":1266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":633,"num_whiteouts":169,"num_read":104651,"num_read_kb":363518,"num_write":7148,"num_write_kb":203466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":47754073,"num_keys_recovered":68,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5508,"ondisk_log_size":5508,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":526828,"kb_used_data":245112,"kb_used_omap":12,"kb_used_meta":281587,"kb_avail":754447892,"statfs":{"total":773094113280,"available":772554641408,"internally_reserved":0,"allocated":250994688,"data_stored":260816630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":288345656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53442883,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":19362,"num_read_kb":25102,"num_write":227,"num_write_kb":14137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.900927"},"pg_stats":[{"pgid":"3.1f","version":"64'160","reported_seq":668,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.171013+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:08:08.171013+0000","last_peered":"2024-09-13T09:08:08.171013+0000","last_clean":"2024-09-13T09:08:08.171013+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:08:08.171013+0000","last_undegraded":"2024-09-13T09:08:08.171013+0000","last_fullsized":"2024-09-13T09:08:08.171013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":10845646,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":843,"num_read_kb":4918,"num_write":213,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"64'208","reported_seq":1695,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.209774+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:08:08.209774+0000","last_peered":"2024-09-13T09:08:08.209774+0000","last_clean":"2024-09-13T09:08:08.209774+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:08:08.209774+0000","last_undegraded":"2024-09-13T09:08:08.209774+0000","last_fullsized":"2024-09-13T09:08:08.209774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":37602897,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":2432,"num_read_kb":9910,"num_write":243,"num_write_kb":3848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"60'179","reported_seq":2312,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.669984+0000","last_change":"2024-09-13T09:07:15.879980+0000","last_active":"2024-09-13T09:08:05.669984+0000","last_peered":"2024-09-13T09:08:05.669984+0000","last_clean":"2024-09-13T09:08:05.669984+0000","last_became_active":"2024-09-13T09:07:13.856004+0000","last_became_peered":"2024-09-13T09:07:13.856004+0000","last_unstale":"2024-09-13T09:08:05.669984+0000","last_undegraded":"2024-09-13T09:08:05.669984+0000","last_fullsized":"2024-09-13T09:08:05.669984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:37.046343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":48416107,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":2124,"num_read_kb":25278,"num_write":253,"num_write_kb":13248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"61'85","reported_seq":1507,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.004737+0000","last_change":"2024-09-13T09:07:51.093764+0000","last_active":"2024-09-13T09:08:06.004737+0000","last_peered":"2024-09-13T09:08:06.004737+0000","last_clean":"2024-09-13T09:08:06.004737+0000","last_became_active":"2024-09-13T09:07:50.836531+0000","last_became_peered":"2024-09-13T09:07:50.836531+0000","last_unstale":"2024-09-13T09:08:06.004737+0000","last_undegraded":"2024-09-13T09:08:06.004737+0000","last_fullsized":"2024-09-13T09:08:06.004737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:11.585290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":46102973,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2515,"num_read_kb":22664,"num_write":119,"num_write_kb":10107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":3944384,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"60'268","reported_seq":17538,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.434612+0000","last_change":"2024-09-13T09:07:16.108727+0000","last_active":"2024-09-13T09:08:05.434612+0000","last_peered":"2024-09-13T09:08:05.434612+0000","last_clean":"2024-09-13T09:08:05.434612+0000","last_became_active":"2024-09-13T09:07:13.862565+0000","last_became_peered":"2024-09-13T09:07:13.862565+0000","last_unstale":"2024-09-13T09:08:05.434612+0000","last_undegraded":"2024-09-13T09:08:05.434612+0000","last_fullsized":"2024-09-13T09:08:05.434612+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:32.305423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":58876863,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":18533,"num_read_kb":25518,"num_write":341,"num_write_kb":16770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"61'180","reported_seq":1945,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-13T09:08:05.963785+0000","last_change":"2024-09-13T09:08:05.964026+0000","last_active":"2024-09-13T09:08:05.963785+0000","last_peered":"2024-09-13T09:08:05.894139+0000","last_clean":"2024-09-13T09:08:05.894139+0000","last_became_active":"2024-09-13T09:07:50.833042+0000","last_became_peered":"2024-09-13T09:07:50.833042+0000","last_unstale":"2024-09-13T09:08:05.963785+0000","last_undegraded":"2024-09-13T09:08:05.963785+0000","last_fullsized":"2024-09-13T09:08:05.963785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:47.212272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":47325778,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":3012,"num_read_kb":13732,"num_write":314,"num_write_kb":16562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":31801546,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"60'93","reported_seq":749,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.175265+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:08:03.175265+0000","last_peered":"2024-09-13T09:08:03.175265+0000","last_clean":"2024-09-13T09:08:03.175265+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:08:03.175265+0000","last_undegraded":"2024-09-13T09:08:03.175265+0000","last_fullsized":"2024-09-13T09:08:03.175265+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":2509931,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":580,"num_read_kb":3310,"num_write":93,"num_write_kb":4753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"59'154","reported_seq":1220,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976447+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:08:06.976447+0000","last_peered":"2024-09-13T09:08:06.976447+0000","last_clean":"2024-09-13T09:08:06.976447+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:08:06.976447+0000","last_undegraded":"2024-09-13T09:08:06.976447+0000","last_fullsized":"2024-09-13T09:08:06.976447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":18577308,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":1714,"num_read_kb":8005,"num_write":153,"num_write_kb":3690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"62'240","reported_seq":2822,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976621+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:08:06.976621+0000","last_peered":"2024-09-13T09:08:06.976621+0000","last_clean":"2024-09-13T09:08:06.976621+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:08:06.976621+0000","last_undegraded":"2024-09-13T09:08:06.976621+0000","last_fullsized":"2024-09-13T09:08:06.976621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":42969160,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3400,"num_read_kb":21858,"num_write":270,"num_write_kb":16106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"61'110","reported_seq":625,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.848566+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:08:05.848566+0000","last_peered":"2024-09-13T09:08:05.848566+0000","last_clean":"2024-09-13T09:08:05.848566+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:08:05.848566+0000","last_undegraded":"2024-09-13T09:08:05.848566+0000","last_fullsized":"2024-09-13T09:08:05.848566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":9332974,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":639,"num_read_kb":3988,"num_write":126,"num_write_kb":3362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"61'205","reported_seq":2005,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.708645+0000","last_change":"2024-09-13T09:07:15.873331+0000","last_active":"2024-09-13T09:08:05.708645+0000","last_peered":"2024-09-13T09:08:05.708645+0000","last_clean":"2024-09-13T09:08:05.708645+0000","last_became_active":"2024-09-13T09:07:13.866813+0000","last_became_peered":"2024-09-13T09:07:13.866813+0000","last_unstale":"2024-09-13T09:08:05.708645+0000","last_undegraded":"2024-09-13T09:08:05.708645+0000","last_fullsized":"2024-09-13T09:08:05.708645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.439602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":45587035,"num_objects":26,"num_object_clones":9,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":2341,"num_read_kb":17771,"num_write":219,"num_write_kb":11683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"61'286","reported_seq":17229,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.058722+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:08:05.058722+0000","last_peered":"2024-09-13T09:08:05.058722+0000","last_clean":"2024-09-13T09:08:05.058722+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:08:05.058722+0000","last_undegraded":"2024-09-13T09:08:05.058722+0000","last_fullsized":"2024-09-13T09:08:05.058722+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":16839736,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":5,"num_read":18600,"num_read_kb":25170,"num_write":260,"num_write_kb":6627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944805+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:08:03.944805+0000","last_peered":"2024-09-13T09:08:03.944805+0000","last_clean":"2024-09-13T09:08:03.944805+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:08:03.944805+0000","last_undegraded":"2024-09-13T09:08:03.944805+0000","last_fullsized":"2024-09-13T09:08:03.944805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"56'352","reported_seq":1373,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.178985+0000","last_change":"2024-09-13T09:07:16.108420+0000","last_active":"2024-09-13T09:08:03.178985+0000","last_peered":"2024-09-13T09:08:03.178985+0000","last_clean":"2024-09-13T09:08:03.178985+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:08:03.178985+0000","last_undegraded":"2024-09-13T09:08:03.178985+0000","last_fullsized":"2024-09-13T09:08:03.178985+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":15010381,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":2411,"num_read_kb":8412,"num_write":455,"num_write_kb":1552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":56,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994953+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:08:02.994953+0000","last_peered":"2024-09-13T09:08:02.994953+0000","last_clean":"2024-09-13T09:08:02.994953+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:08:02.994953+0000","last_undegraded":"2024-09-13T09:08:02.994953+0000","last_fullsized":"2024-09-13T09:08:02.994953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"56'80","reported_seq":976,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994878+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:08:02.994878+0000","last_peered":"2024-09-13T09:08:02.994878+0000","last_clean":"2024-09-13T09:08:02.994878+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:08:02.994878+0000","last_undegraded":"2024-09-13T09:08:02.994878+0000","last_fullsized":"2024-09-13T09:08:02.994878+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":11264256,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":903,"num_read_kb":10673,"num_write":81,"num_write_kb":3674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944836+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:08:03.944836+0000","last_peered":"2024-09-13T09:08:03.944836+0000","last_clean":"2024-09-13T09:08:03.944836+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:08:03.944836+0000","last_undegraded":"2024-09-13T09:08:03.944836+0000","last_fullsized":"2024-09-13T09:08:03.944836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"56'256","reported_seq":675,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.174506+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:08:03.174506+0000","last_peered":"2024-09-13T09:08:03.174506+0000","last_clean":"2024-09-13T09:08:03.174506+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:08:03.174506+0000","last_undegraded":"2024-09-13T09:08:03.174506+0000","last_fullsized":"2024-09-13T09:08:03.174506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":18142525,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1343,"num_read_kb":5247,"num_write":404,"num_write_kb":1434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976644+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:08:06.976644+0000","last_peered":"2024-09-13T09:08:06.976644+0000","last_clean":"2024-09-13T09:08:06.976644+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:08:06.976644+0000","last_undegraded":"2024-09-13T09:08:06.976644+0000","last_fullsized":"2024-09-13T09:08:06.976644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"61'186","reported_seq":1954,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.014724+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:08:06.014724+0000","last_peered":"2024-09-13T09:08:06.014724+0000","last_clean":"2024-09-13T09:08:06.014724+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:08:06.014724+0000","last_undegraded":"2024-09-13T09:08:06.014724+0000","last_fullsized":"2024-09-13T09:08:06.014724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":23622460,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2392,"num_read_kb":12629,"num_write":288,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"61'109","reported_seq":1097,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709401+0000","last_change":"2024-09-13T09:07:15.873298+0000","last_active":"2024-09-13T09:08:05.709401+0000","last_peered":"2024-09-13T09:08:05.709401+0000","last_clean":"2024-09-13T09:08:05.709401+0000","last_became_active":"2024-09-13T09:07:13.852418+0000","last_became_peered":"2024-09-13T09:07:13.852418+0000","last_unstale":"2024-09-13T09:08:05.709401+0000","last_undegraded":"2024-09-13T09:08:05.709401+0000","last_fullsized":"2024-09-13T09:08:05.709401+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:58.473601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":22810773,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":1552,"num_read_kb":7181,"num_write":120,"num_write_kb":957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":59,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944653+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:08:03.944653+0000","last_peered":"2024-09-13T09:08:03.944653+0000","last_clean":"2024-09-13T09:08:03.944653+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:08:03.944653+0000","last_undegraded":"2024-09-13T09:08:03.944653+0000","last_fullsized":"2024-09-13T09:08:03.944653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"56'98","reported_seq":1538,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976282+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:08:06.976282+0000","last_peered":"2024-09-13T09:08:06.976282+0000","last_clean":"2024-09-13T09:08:06.976282+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:08:06.976282+0000","last_undegraded":"2024-09-13T09:08:06.976282+0000","last_fullsized":"2024-09-13T09:08:06.976282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":11014388,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":1444,"num_read_kb":10181,"num_write":100,"num_write_kb":4780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.938292+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:08:03.938292+0000","last_peered":"2024-09-13T09:08:03.938292+0000","last_clean":"2024-09-13T09:08:03.938292+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:08:03.938292+0000","last_undegraded":"2024-09-13T09:08:03.938292+0000","last_fullsized":"2024-09-13T09:08:03.938292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"61'103","reported_seq":922,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:04.218363+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:08:04.218363+0000","last_peered":"2024-09-13T09:08:04.218363+0000","last_clean":"2024-09-13T09:08:04.218363+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:08:04.218363+0000","last_undegraded":"2024-09-13T09:08:04.218363+0000","last_fullsized":"2024-09-13T09:08:04.218363+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":6024682,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1019,"num_read_kb":6803,"num_write":96,"num_write_kb":3261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.943334+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:08:03.943334+0000","last_peered":"2024-09-13T09:08:03.943334+0000","last_clean":"2024-09-13T09:08:03.943334+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:08:03.943334+0000","last_undegraded":"2024-09-13T09:08:03.943334+0000","last_fullsized":"2024-09-13T09:08:03.943334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"56'130","reported_seq":309,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.994707+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:08:02.994707+0000","last_peered":"2024-09-13T09:08:02.994707+0000","last_clean":"2024-09-13T09:08:02.994707+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:08:02.994707+0000","last_undegraded":"2024-09-13T09:08:02.994707+0000","last_fullsized":"2024-09-13T09:08:02.994707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":1465988,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":473,"num_read_kb":3329,"num_write":175,"num_write_kb":1460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":58,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944810+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:08:03.944810+0000","last_peered":"2024-09-13T09:08:03.944810+0000","last_clean":"2024-09-13T09:08:03.944810+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:08:03.944810+0000","last_undegraded":"2024-09-13T09:08:03.944810+0000","last_fullsized":"2024-09-13T09:08:03.944810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:04.218502+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:08:04.218502+0000","last_peered":"2024-09-13T09:08:04.218502+0000","last_clean":"2024-09-13T09:08:04.218502+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:08:04.218502+0000","last_undegraded":"2024-09-13T09:08:04.218502+0000","last_fullsized":"2024-09-13T09:08:04.218502+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"57'153","reported_seq":1984,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.178847+0000","last_change":"2024-09-13T09:07:16.108021+0000","last_active":"2024-09-13T09:08:03.178847+0000","last_peered":"2024-09-13T09:08:03.178847+0000","last_clean":"2024-09-13T09:08:03.178847+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:08:03.178847+0000","last_undegraded":"2024-09-13T09:08:03.178847+0000","last_fullsized":"2024-09-13T09:08:03.178847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":26026737,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2139,"num_read_kb":12139,"num_write":296,"num_write_kb":7443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"61'220","reported_seq":2078,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.705592+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:08:05.705592+0000","last_peered":"2024-09-13T09:08:05.705592+0000","last_clean":"2024-09-13T09:08:05.705592+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:08:05.705592+0000","last_undegraded":"2024-09-13T09:08:05.705592+0000","last_fullsized":"2024-09-13T09:08:05.705592+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":33348128,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":2951,"num_read_kb":10450,"num_write":297,"num_write_kb":4571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"61'150","reported_seq":704,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.706520+0000","last_change":"2024-09-13T09:07:53.864337+0000","last_active":"2024-09-13T09:08:05.706520+0000","last_peered":"2024-09-13T09:08:05.706520+0000","last_clean":"2024-09-13T09:08:05.706520+0000","last_became_active":"2024-09-13T09:07:53.862992+0000","last_became_peered":"2024-09-13T09:07:53.862992+0000","last_unstale":"2024-09-13T09:08:05.706520+0000","last_undegraded":"2024-09-13T09:08:05.706520+0000","last_fullsized":"2024-09-13T09:08:05.706520+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:14.717209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":33289951,"num_objects":24,"num_object_clones":8,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":480,"num_read_kb":5891,"num_write":155,"num_write_kb":7563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":2632291,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"56'70","reported_seq":1269,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.938382+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:08:03.938382+0000","last_peered":"2024-09-13T09:08:03.938382+0000","last_clean":"2024-09-13T09:08:03.938382+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:08:03.938382+0000","last_undegraded":"2024-09-13T09:08:03.938382+0000","last_fullsized":"2024-09-13T09:08:03.938382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":8722868,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1189,"num_read_kb":13116,"num_write":94,"num_write_kb":8052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"61'90","reported_seq":15663,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.848616+0000","last_change":"2024-09-13T09:07:15.873583+0000","last_active":"2024-09-13T09:08:05.848616+0000","last_peered":"2024-09-13T09:08:05.848616+0000","last_clean":"2024-09-13T09:08:05.848616+0000","last_became_active":"2024-09-13T09:07:13.859769+0000","last_became_peered":"2024-09-13T09:07:13.859769+0000","last_unstale":"2024-09-13T09:08:05.848616+0000","last_undegraded":"2024-09-13T09:08:05.848616+0000","last_fullsized":"2024-09-13T09:08:05.848616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:52.161129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":40221705,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":15551,"num_read_kb":23982,"num_write":139,"num_write_kb":12633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"61'63","reported_seq":1216,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709177+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:08:05.709177+0000","last_peered":"2024-09-13T09:08:05.709177+0000","last_clean":"2024-09-13T09:08:05.709177+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:08:05.709177+0000","last_undegraded":"2024-09-13T09:08:05.709177+0000","last_fullsized":"2024-09-13T09:08:05.709177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":9877649,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1029,"num_read_kb":5678,"num_write":63,"num_write_kb":1398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"61'72","reported_seq":2147,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:05.709780+0000","last_change":"2024-09-13T09:07:15.859429+0000","last_active":"2024-09-13T09:08:05.709780+0000","last_peered":"2024-09-13T09:08:05.709780+0000","last_clean":"2024-09-13T09:08:05.709780+0000","last_became_active":"2024-09-13T09:07:13.855153+0000","last_became_peered":"2024-09-13T09:07:13.855153+0000","last_unstale":"2024-09-13T09:08:05.709780+0000","last_undegraded":"2024-09-13T09:08:05.709780+0000","last_fullsized":"2024-09-13T09:08:05.709780+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:15.261153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":21285344,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":2044,"num_read_kb":9681,"num_write":69,"num_write_kb":1724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"56'120","reported_seq":1346,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.974106+0000","last_change":"2024-09-13T09:07:51.052606+0000","last_active":"2024-09-13T09:08:03.974106+0000","last_peered":"2024-09-13T09:08:03.974106+0000","last_clean":"2024-09-13T09:08:03.974106+0000","last_became_active":"2024-09-13T09:07:50.848333+0000","last_became_peered":"2024-09-13T09:07:50.848333+0000","last_unstale":"2024-09-13T09:08:03.974106+0000","last_undegraded":"2024-09-13T09:08:03.974106+0000","last_fullsized":"2024-09-13T09:08:03.974106+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:23:05.120217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":12634663,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":5,"num_read":1522,"num_read_kb":4341,"num_write":119,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":885530,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"61'163","reported_seq":1546,"reported_epoch":62,"state":"peering","last_fresh":"2024-09-13T09:08:05.963741+0000","last_change":"2024-09-13T09:08:05.964022+0000","last_active":"2024-09-13T09:08:05.963741+0000","last_peered":"2024-09-13T09:08:05.893968+0000","last_clean":"2024-09-13T09:08:05.893968+0000","last_became_active":"2024-09-13T09:07:53.861512+0000","last_became_peered":"2024-09-13T09:07:53.861512+0000","last_unstale":"2024-09-13T09:08:05.963741+0000","last_undegraded":"2024-09-13T09:08:05.963741+0000","last_fullsized":"2024-09-13T09:08:05.963741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:46:53.676923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":46116942,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":1444,"num_read_kb":13900,"num_write":179,"num_write_kb":7740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":6431007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"62'423","reported_seq":1645,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.028591+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:08:06.028591+0000","last_peered":"2024-09-13T09:08:06.028591+0000","last_clean":"2024-09-13T09:08:06.028591+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:08:06.028591+0000","last_undegraded":"2024-09-13T09:08:06.028591+0000","last_fullsized":"2024-09-13T09:08:06.028591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":15305898,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":4,"num_read":3517,"num_read_kb":4793,"num_write":694,"num_write_kb":3399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"56'192","reported_seq":1933,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T09:08:02.930510+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:08:02.930510+0000","last_peered":"2024-09-13T09:08:02.930510+0000","last_clean":"2024-09-13T09:08:02.930510+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:08:02.930510+0000","last_undegraded":"2024-09-13T09:08:02.930510+0000","last_fullsized":"2024-09-13T09:08:02.930510+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":27927802,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":2773,"num_read_kb":7288,"num_write":288,"num_write_kb":4024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"56'190","reported_seq":909,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-13T09:08:03.944685+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:08:03.944685+0000","last_peered":"2024-09-13T09:08:03.944685+0000","last_clean":"2024-09-13T09:08:03.944685+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:08:03.944685+0000","last_undegraded":"2024-09-13T09:08:03.944685+0000","last_fullsized":"2024-09-13T09:08:03.944685+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":4364767,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":1626,"num_read_kb":5582,"num_write":271,"num_write_kb":4745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":773464315,"num_objects":629,"num_object_clones":160,"num_object_copies":1258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":629,"num_whiteouts":169,"num_read":104535,"num_read_kb":363418,"num_write":6987,"num_write_kb":201048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":47163705,"num_keys_recovered":68,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":254537728,"data_stored":267497534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5388,"ondisk_log_size":5388,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60208,"kb_used_data":24996,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94311632,"statfs":{"total":96636764160,"available":96575111168,"internally_reserved":0,"allocated":25595904,"data_stored":25969500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.88700000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.194},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.169}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.286}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0389999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72948,"kb_used_data":37740,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94298892,"statfs":{"total":96636764160,"available":96562065408,"internally_reserved":0,"allocated":38645760,"data_stored":41642616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.377}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.42499999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.44400000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.624}]}]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52452,"kb_used_data":17224,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94319388,"statfs":{"total":96636764160,"available":96583053312,"internally_reserved":0,"allocated":17637376,"data_stored":17831540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.56999999999999995},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.67300000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60632,"kb_used_data":29520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94311208,"statfs":{"total":96636764160,"available":96574676992,"internally_reserved":0,"allocated":30228480,"data_stored":29638714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":2.2650000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.309}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.874},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.57699999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75200,"kb_used_data":44084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94296640,"statfs":{"total":96636764160,"available":96559759360,"internally_reserved":0,"allocated":45142016,"data_stored":50164914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.37}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.80900000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55760,"kb_used_data":20540,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94316080,"statfs":{"total":96636764160,"available":96579665920,"internally_reserved":0,"allocated":21032960,"data_stored":24108649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.72599999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.79000000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.89700000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.871},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.99299999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1459999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73184,"kb_used_data":33868,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94298656,"statfs":{"total":96636764160,"available":96561823744,"internally_reserved":0,"allocated":34680832,"data_stored":34068846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.501},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.748}]},{"osd":2,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.622},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.97999999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82799999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.85199999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76444,"kb_used_data":37140,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94295396,"statfs":{"total":96636764160,"available":96558485504,"internally_reserved":0,"allocated":38031360,"data_stored":37391851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.501},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.9740000000000002,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.9050000000000002,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":8.0850000000000009,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44429312,"data_stored":49848197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31068160,"data_stored":30865820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19062784,"data_stored":19645440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41340928,"data_stored":44720649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33980416,"data_stored":33757952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":39424000,"data_stored":39189561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20348928,"data_stored":23817132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24883200,"data_stored":25652783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:08:10.616 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:10.613+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:10.684 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:10.681+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:10.694 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:10.694 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:10.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:10.785 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:10.779+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:10.879 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:10.879 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:10.886 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:10.881+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:10.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:10.987 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:10.981+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:11.072 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:11.073 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:11.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:11.088 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:11.081+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:11.181 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone20 from 0x22c499c (overlap 0x1b7a680) to 0x2bf70c 2024-09-13T09:08:11.188 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:11.181+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:11.288 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:11.285+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:11.293 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:11.293 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:11.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:11.389 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:11.383+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:11.461 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:11.461 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:11.478 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:11.490 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:11.485+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:11.592 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:11.585+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:11.637 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:11.637 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:11.652 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:11.692 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:11.685+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:11.789 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:11.789 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:11.793 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:11.789+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:11.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:11.842 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-13T09:08:11.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:08:11.893 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:11.887+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:11.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:12.015 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:12.011+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:12.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:12.096 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:12.089+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:12.106 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:12.106 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:12.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:12.211 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:12.207+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:12.211 INFO:teuthology.orchestra.run.smithi031.stdout:2199 flatten 2024-09-13T09:08:12.297 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:12.291+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:12.319 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:12.319 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:12.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:12.397 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:12.393+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:12.420 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:12.420 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:12.440 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:12.440 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:12.453 INFO:teuthology.orchestra.run.smithi031.stdout:2200 write 0x31d3ab2 thru 0x31d4059 (0x5a8 bytes) 2024-09-13T09:08:12.458 INFO:teuthology.orchestra.run.smithi031.stdout:2202 trunc from 0x31d405a to 0x4870770 2024-09-13T09:08:12.461 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:12.461 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:12.465 INFO:teuthology.orchestra.run.smithi031.stdout:2203 writesame 0x16d1eab thru 0x16dd32b data_size 0x3(0xb481 bytes) 2024-09-13T09:08:12.476 INFO:teuthology.orchestra.run.smithi031.stdout:2204 punch from 0x3522eef to 0x3526211, (0x3322 bytes) 2024-09-13T09:08:12.476 INFO:teuthology.orchestra.run.smithi031.stdout:2205 writesame 0x8f1fcb thru 0x900852 data_size 0x1c(0xe888 bytes) 2024-09-13T09:08:12.483 INFO:teuthology.orchestra.run.smithi031.stdout:2207 read 0x96fc20 thru 0x97a27a (0xa65b bytes) 2024-09-13T09:08:12.483 INFO:teuthology.orchestra.run.smithi031.stdout:2209 punch from 0x35dff9d to 0x35ea182, (0xa1e5 bytes) 2024-09-13T09:08:12.495 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:12.495 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:12.498 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:12.493+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:12.500 INFO:teuthology.orchestra.run.smithi031.stdout:2211 write 0xc7756d1 thru 0xc77ccdd (0x760d bytes) 2024-09-13T09:08:12.504 INFO:teuthology.orchestra.run.smithi031.stdout:2212 punch from 0x889dac4 to 0x88a558d, (0x7ac9 bytes) 2024-09-13T09:08:12.505 INFO:teuthology.orchestra.run.smithi031.stdout:2213 read 0x837f85b thru 0x838a22b (0xa9d1 bytes) 2024-09-13T09:08:12.505 INFO:teuthology.orchestra.run.smithi031.stdout:2214 read 0x2d6f83 thru 0x2e4992 (0xda10 bytes) 2024-09-13T09:08:12.505 INFO:teuthology.orchestra.run.smithi031.stdout:2215 writesame 0x68314e2 thru 0x683484b data_size 0x1(0x336a bytes) 2024-09-13T09:08:12.517 INFO:teuthology.orchestra.run.smithi031.stdout:2216 read 0x1a0fe56 thru 0x1a186a1 (0x884c bytes) 2024-09-13T09:08:12.518 INFO:teuthology.orchestra.run.smithi031.stdout:2219 punch from 0x780d09a to 0x781cadf, (0xfa45 bytes) 2024-09-13T09:08:12.518 INFO:teuthology.orchestra.run.smithi031.stdout:2220 write 0x604152 thru 0x6094de (0x538d bytes) 2024-09-13T09:08:12.521 INFO:teuthology.orchestra.run.smithi031.stdout:2221 read 0x2c05246 thru 0x2c07852 (0x260d bytes) 2024-09-13T09:08:12.521 INFO:teuthology.orchestra.run.smithi031.stdout:2222 writesame 0xa0c82a3 thru 0xa0d3d0b data_size 0x1(0xba69 bytes) 2024-09-13T09:08:12.521 INFO:teuthology.orchestra.run.smithi031.stdout:2223 punch from 0x61fcb94 to 0x620b310, (0xe77c bytes) 2024-09-13T09:08:12.522 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:12.522 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:12.523 INFO:teuthology.orchestra.run.smithi031.stdout:2224 trunc from 0xc77ccde to 0xd9e7387 2024-09-13T09:08:12.526 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:12.526 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:12.526 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:12.526 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:12.530 INFO:teuthology.orchestra.run.smithi031.stdout:2225 punch from 0xb1758b1 to 0xb17aeac, (0x55fb bytes) 2024-09-13T09:08:12.530 INFO:teuthology.orchestra.run.smithi031.stdout:2226 punch from 0x6454ead to 0x645846e, (0x35c1 bytes) 2024-09-13T09:08:12.530 INFO:teuthology.orchestra.run.smithi031.stdout:2227 writesame 0x19fb61d thru 0x19fc733 data_size 0x5(0x1117 bytes) 2024-09-13T09:08:12.535 INFO:teuthology.orchestra.run.smithi031.stdout:2228 write 0x5d02459 thru 0x5d066ea (0x4292 bytes) 2024-09-13T09:08:12.539 INFO:teuthology.orchestra.run.smithi031.stdout:2232 read 0x5beb5ca thru 0x5bf98e7 (0xe31e bytes) 2024-09-13T09:08:12.544 INFO:teuthology.orchestra.run.smithi031.stdout:2233 trunc from 0xd9e7387 to 0xa1ea268 2024-09-13T09:08:12.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:12.549 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:12.549 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:12.558 INFO:teuthology.orchestra.run.smithi031.stdout:2234 writesame 0x9b97ef6 thru 0x9ba3d89 data_size 0x2(0xbe94 bytes) 2024-09-13T09:08:12.573 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:12.573 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:12.574 INFO:teuthology.orchestra.run.smithi031.stdout:2237 read 0x2ae1cce thru 0x2ae59a4 (0x3cd7 bytes) 2024-09-13T09:08:12.575 INFO:teuthology.orchestra.run.smithi031.stdout:2238 punch from 0x77425f1 to 0x7748d07, (0x6716 bytes) 2024-09-13T09:08:12.575 INFO:teuthology.orchestra.run.smithi031.stdout:2239 punch from 0x2ee0981 to 0x2ee3be6, (0x3265 bytes) 2024-09-13T09:08:12.581 INFO:teuthology.orchestra.run.smithi031.stdout:2241 trunc from 0xa1ea268 to 0xe5c899b 2024-09-13T09:08:12.588 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:12.588 INFO:teuthology.orchestra.run.smithi031.stdout:2243 punch from 0x242e8d3 to 0x24358b6, (0x6fe3 bytes) 2024-09-13T09:08:12.588 INFO:teuthology.orchestra.run.smithi031.stdout:2246 write 0xb8c0c15 thru 0xb8c5602 (0x49ee bytes) 2024-09-13T09:08:12.594 INFO:teuthology.orchestra.run.smithi031.stdout:2247 write 0x793f8b6 thru 0x794217f (0x28ca bytes) 2024-09-13T09:08:12.596 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:12.596 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:12.599 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:12.595+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:12.600 INFO:teuthology.orchestra.run.smithi031.stdout:2248 writesame 0x6b935f5 thru 0x6b9f694 data_size 0x86(0xc0a0 bytes) 2024-09-13T09:08:12.606 INFO:teuthology.orchestra.run.smithi031.stdout:2249 writesame 0xdfe470f thru 0xdfe6912 data_size 0xe(0x2204 bytes) 2024-09-13T09:08:12.611 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:12.612 INFO:teuthology.orchestra.run.smithi031.stdout:2251 trunc from 0xe5c899b to 0xda64bcf 2024-09-13T09:08:12.623 INFO:teuthology.orchestra.run.smithi031.stdout:2252 read 0x18977b8 thru 0x18a0ac6 (0x930f bytes) 2024-09-13T09:08:12.624 INFO:teuthology.orchestra.run.smithi031.stdout:2254 write 0x76c56b7 thru 0x76d5620 (0xff6a bytes) 2024-09-13T09:08:12.628 INFO:teuthology.orchestra.run.smithi031.stdout:2255 read 0xba625f1 thru 0xba6da2b (0xb43b bytes) 2024-09-13T09:08:12.636 INFO:teuthology.orchestra.run.smithi031.stdout:2256 trunc from 0xda64bcf to 0x84010a6 2024-09-13T09:08:12.649 INFO:teuthology.orchestra.run.smithi031.stdout:2257 punch from 0x7c9212e to 0x7c9a50c, (0x83de bytes) 2024-09-13T09:08:12.649 INFO:teuthology.orchestra.run.smithi031.stdout:2260 read 0x5d319bf thru 0x5d35619 (0x3c5b bytes) 2024-09-13T09:08:12.650 INFO:teuthology.orchestra.run.smithi031.stdout:2261 read 0xb70032 thru 0xb71f11 (0x1ee0 bytes) 2024-09-13T09:08:12.650 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:12.650 INFO:teuthology.orchestra.run.smithi031.stdout:2264 trunc from 0x84010a6 to 0x82fb461 2024-09-13T09:08:12.656 INFO:teuthology.orchestra.run.smithi031.stdout:2269 punch from 0x1a6e3f9 to 0x1a701a1, (0x1da8 bytes) 2024-09-13T09:08:12.656 INFO:teuthology.orchestra.run.smithi031.stdout:2270 read 0x3f30629 thru 0x3f38c06 (0x85de bytes) 2024-09-13T09:08:12.656 INFO:teuthology.orchestra.run.smithi031.stdout:2271 write 0x7ca8546 thru 0x7cab69a (0x3155 bytes) 2024-09-13T09:08:12.660 INFO:teuthology.orchestra.run.smithi031.stdout:2272 read 0x184db94 thru 0x185486c (0x6cd9 bytes) 2024-09-13T09:08:12.660 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:12.670 INFO:teuthology.orchestra.run.smithi031.stdout:2274 writesame 0xae146fa thru 0xae1bd01 data_size 0xc(0x7608 bytes) 2024-09-13T09:08:12.676 INFO:teuthology.orchestra.run.smithi031.stdout:2275 trunc from 0xae1bd02 to 0xea9a24f 2024-09-13T09:08:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:12.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:12.681 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:12.681 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:12.681 INFO:teuthology.orchestra.run.smithi031.stdout:2276 writesame 0x71aee19 thru 0x71b80e9 data_size 0x1(0x92d1 bytes) 2024-09-13T09:08:12.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:12.702 INFO:teuthology.orchestra.run.smithi031.stdout:2277 writesame 0xb72048 thru 0xb7f985 data_size 0x5d(0xd93e bytes) 2024-09-13T09:08:12.702 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:12.695+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:12.707 INFO:teuthology.orchestra.run.smithi031.stdout:2279 read 0x79e6f85 thru 0x79ecb73 (0x5bef bytes) 2024-09-13T09:08:12.707 INFO:teuthology.orchestra.run.smithi031.stdout:2280 write 0xdcca3cc thru 0xdcd4552 (0xa187 bytes) 2024-09-13T09:08:12.712 INFO:teuthology.orchestra.run.smithi031.stdout:2281 read 0xd1e2c25 thru 0xd1f2c0b (0xffe7 bytes) 2024-09-13T09:08:12.728 INFO:teuthology.orchestra.run.smithi031.stdout:2282 trunc from 0xea9a24f to 0x15d8eb5 2024-09-13T09:08:12.755 INFO:teuthology.orchestra.run.smithi031.stdout:2284 writesame 0x21141d9 thru 0x212407c data_size 0x56(0xfea4 bytes) 2024-09-13T09:08:12.755 INFO:teuthology.orchestra.run.smithi031.stdout:2285 read 0x16f1adf thru 0x16f3d61 (0x2283 bytes) 2024-09-13T09:08:12.776 INFO:teuthology.orchestra.run.smithi031.stdout:2286 write 0xd9f3a0e thru 0xd9f5481 (0x1a74 bytes) 2024-09-13T09:08:12.780 INFO:teuthology.orchestra.run.smithi031.stdout:2287 write 0x6c09ea0 thru 0x6c12443 (0x85a4 bytes) 2024-09-13T09:08:12.784 INFO:teuthology.orchestra.run.smithi031.stdout:2289 write 0x70027bf thru 0x700a69a (0x7edc bytes) 2024-09-13T09:08:12.789 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:12.789 INFO:teuthology.orchestra.run.smithi031.stdout:2291 read 0x9d1d86c thru 0x9d2999b (0xc130 bytes) 2024-09-13T09:08:12.790 INFO:teuthology.orchestra.run.smithi031.stdout:2293 punch from 0xb99cdfc to 0xb9a9494, (0xc698 bytes) 2024-09-13T09:08:12.790 INFO:teuthology.orchestra.run.smithi031.stdout:2294 write 0x6cfc284 thru 0x6cfcd27 (0xaa4 bytes) 2024-09-13T09:08:12.794 INFO:teuthology.orchestra.run.smithi031.stdout:2295 read 0x41fa263 thru 0x41fd29e (0x303c bytes) 2024-09-13T09:08:12.794 INFO:teuthology.orchestra.run.smithi031.stdout:2296 writesame 0x7501f37 thru 0x7504826 data_size 0x28(0x28f0 bytes) 2024-09-13T09:08:12.797 INFO:teuthology.orchestra.run.smithi031.stdout:2298 read 0x708484d thru 0x70854b5 (0xc69 bytes) 2024-09-13T09:08:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:2299 trunc from 0xd9f5482 to 0xbd783e5 2024-09-13T09:08:12.800 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:12.795+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:12.819 INFO:teuthology.orchestra.run.smithi031.stdout:2300 trunc from 0xbd783e5 to 0x77a476f 2024-09-13T09:08:12.826 INFO:teuthology.orchestra.run.smithi031.stdout:2302 read 0x6619ce2 thru 0x662077e (0x6a9d bytes) 2024-09-13T09:08:12.826 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:12.826 INFO:teuthology.orchestra.run.smithi031.stdout:2304 write 0xbd8c9a thru 0xbdf55a (0x68c1 bytes) 2024-09-13T09:08:12.828 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:12.828 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487598925999997, 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:08:12.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487582152999998, 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:08:12.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.837 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487544847999999, 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:08:12.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487525134000002, 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:08:12.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:12.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487508808999998, 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:08:12.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487492942999999, 2024-09-13T09:08:12.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:08:12.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487477157999997, 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:08:12.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487458087, 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:08:12.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:08:12.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487441539999999, 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:12.867 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487413941, 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:08:12.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487396699000001, 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.872 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487366256000001, 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.876 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:12.880 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487347341000003, 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:08:12.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:08:12.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487329553000002, 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:08:12.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:08:12.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487304328999997, 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487286124999997, 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:12.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:08:12.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487265807, 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:12.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487247259, 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:08:12.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:08:12.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487229984999999, 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:08:12.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:12.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.487212530999997, 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:08:12.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:12.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:12.916 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:12.916 INFO:teuthology.orchestra.run.smithi031.stdout:2305 trunc from 0x77a476f to 0x53e3469 2024-09-13T09:08:12.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:12.918 INFO:teuthology.orchestra.run.smithi031.stdout:2307 punch from 0x2e95b5b to 0x2e96e31, (0x12d6 bytes) 2024-09-13T09:08:12.918 INFO:teuthology.orchestra.run.smithi031.stdout:2309 writesame 0xcda339d thru 0xcdb1f50 data_size 0x1c(0xebb4 bytes) 2024-09-13T09:08:12.918 INFO:teuthology.orchestra.run.smithi031.stdout:2310 trunc from 0xcdb1f51 to 0x4d347d6 2024-09-13T09:08:12.919 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:12.919 INFO:teuthology.orchestra.run.smithi031.stdout:2312 writesame 0xe0fe75a thru 0xe1043df data_size 0xd(0x5c86 bytes) 2024-09-13T09:08:12.919 INFO:teuthology.orchestra.run.smithi031.stdout:2313 read 0xdb9b0e5 thru 0xdba1e7b (0x6d97 bytes) 2024-09-13T09:08:12.919 INFO:teuthology.orchestra.run.smithi031.stdout:2314 trunc from 0xe1043e0 to 0xd0090dd 2024-09-13T09:08:12.919 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:12.911+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:12.919 INFO:teuthology.orchestra.run.smithi031.stdout:2315 punch from 0x7253079 to 0x7261178, (0xe0ff bytes) 2024-09-13T09:08:12.920 INFO:teuthology.orchestra.run.smithi031.stdout:2319 write 0x7876ba6 thru 0x78792c8 (0x2723 bytes) 2024-09-13T09:08:12.923 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:12.923 INFO:teuthology.orchestra.run.smithi031.stdout:2323 write 0x2259f99 thru 0x225beda (0x1f42 bytes) 2024-09-13T09:08:12.929 INFO:teuthology.orchestra.run.smithi031.stdout:2324 trunc from 0xd0090dd to 0xd73b189 2024-09-13T09:08:12.937 INFO:teuthology.orchestra.run.smithi031.stdout:2326 read 0x989a2d6 thru 0x989a3d2 (0xfd bytes) 2024-09-13T09:08:12.937 INFO:teuthology.orchestra.run.smithi031.stdout:2327 punch from 0x6f7ae45 to 0x6f83577, (0x8732 bytes) 2024-09-13T09:08:12.937 INFO:teuthology.orchestra.run.smithi031.stdout:2329 clone 22 order 24 su 8388608 sc 14 2024-09-13T09:08:13.018 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:13.013+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:13.021 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:08:13.061 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:13.061 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:13.061 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:13.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:13.123 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:13.119+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:13.219 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:13.215+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:13.242 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:13.243 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:13.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:13.309 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-13T09:08:13.320 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:13.313+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:13.323 INFO:teuthology.orchestra.run.smithi031.stdout:2330 read 0xb6f1bdc thru 0xb700e02 (0xf227 bytes) 2024-09-13T09:08:13.323 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.323 INFO:teuthology.orchestra.run.smithi031.stdout:2332 write 0x25dcb40 thru 0x25de0b7 (0x1578 bytes) 2024-09-13T09:08:13.335 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.335 INFO:teuthology.orchestra.run.smithi031.stdout:2336 read 0x63eec18 thru 0x63faf34 (0xc31d bytes) 2024-09-13T09:08:13.335 INFO:teuthology.orchestra.run.smithi031.stdout:2338 write 0x72f0a7c thru 0x72fe95f (0xdee4 bytes) 2024-09-13T09:08:13.379 INFO:teuthology.orchestra.run.smithi031.stdout:2339 trunc from 0xd73b189 to 0xdb2a2a6 2024-09-13T09:08:13.385 INFO:teuthology.orchestra.run.smithi031.stdout:2341 write 0x1f93257 thru 0x1f95f7b (0x2d25 bytes) 2024-09-13T09:08:13.391 INFO:teuthology.orchestra.run.smithi031.stdout:2342 write 0x58280d6 thru 0x583268f (0xa5ba bytes) 2024-09-13T09:08:13.415 INFO:teuthology.orchestra.run.smithi031.stdout:2343 trunc from 0xdb2a2a6 to 0x15467d1 2024-09-13T09:08:13.421 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:13.415+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:13.436 INFO:teuthology.orchestra.run.smithi031.stdout:2344 read 0x139ca21 thru 0x13ab863 (0xee43 bytes) 2024-09-13T09:08:13.437 INFO:teuthology.orchestra.run.smithi031.stdout:2345 write 0x2157e1 thru 0x220c3b (0xb45b bytes) 2024-09-13T09:08:13.439 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.439 INFO:teuthology.orchestra.run.smithi031.stdout:2348 read 0xf9da94 thru 0xfa9602 (0xbb6f bytes) 2024-09-13T09:08:13.443 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:13.443 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.311719513, 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:08:13.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:08:13.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:13.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.483437333000001, 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:08:13.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:08:13.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56973 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4501504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.194532237000001, 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091397946999999993, 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56973 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.456 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247918+0000", 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247921+0000", 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247927+0000", 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2389999999999999e-06 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247932+0000", 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339077+0000", 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091144366000000004 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339203+0000", 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126919 2024-09-13T09:08:13.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339317+0000", 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113205 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56975 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4515840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.19433847, 2024-09-13T09:08:13.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091417921999999999, 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56975 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247936+0000", 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999824 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248117+0000", 2024-09-13T09:08:13.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180827 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248127+0000", 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248133+0000", 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.19e-06 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339347+0000", 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091213581000000002 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339419+0000", 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.203e-05 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339530+0000", 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111359 2024-09-13T09:08:13.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56976 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3681280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.194295072999999, 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091677039000000002, 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56976 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248155+0000", 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248159+0000", 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248166+0000", 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999999e-06 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248172+0000", 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339558+0000", 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091385818999999993 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339705+0000", 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014693599999999999 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339833+0000", 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012808200000000001 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56990 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3795968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.191154571, 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091546566999999995, 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56990 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251298+0000", 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.041e-06 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251303+0000", 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251319+0000", 2024-09-13T09:08:13.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5852e-05 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342591+0000", 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091271605000000006 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342728+0000", 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013662799999999999 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342843+0000", 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115442 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57022 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4334592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.191071944000001, 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091691988000000002, 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57022 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:13.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251382+0000", 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5659999999999998e-06 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251391+0000", 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0170000000000002e-06 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251397+0000", 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.908e-06 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342870+0000", 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091473410000000005 2024-09-13T09:08:13.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342945+0000", 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4175e-05 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343071+0000", 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012638499999999999 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57026 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5113856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.191049007, 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091880383999999996, 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57026 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251404+0000", 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251410+0000", 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251415+0000", 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2129999999999999e-06 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343098+0000", 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091683073000000004 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343171+0000", 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3149e-05 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343282+0000", 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110899 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57027 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4348928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.191031364000001, 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092079082000000007, 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57027 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251419+0000", 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251421+0000", 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251427+0000", 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4419999999999997e-06 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251432+0000", 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7620000000000003e-06 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343310+0000", 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091878572000000006 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343383+0000", 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2368999999999998e-05 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343499+0000", 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116142 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57028 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5128192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.191014019000001, 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092372671000000003, 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57028 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251439+0000", 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1730000000000002e-06 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251445+0000", 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251450+0000", 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343526+0000", 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092076621999999997 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343605+0000", 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8830000000000002e-05 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343810+0000", 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204544 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.190997495, 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:08:13.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.190917697, 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:13.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.190668626000001, 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:08:13.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:08:13.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.190355485, 2024-09-13T09:08:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:08:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:13.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:08:13.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:08:13.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.189902253, 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:08:13.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.189877739, 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:08:13.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.18961251, 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.520 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.189324661000001, 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:08:13.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57037 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.188888459999999, 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092241045999999993, 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57037 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253152+0000", 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995894 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253567+0000", 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000414886 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253576+0000", 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6750000000000008e-06 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253582+0000", 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345610+0000", 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092027943000000001 2024-09-13T09:08:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345682+0000", 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2311999999999995e-05 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345804+0000", 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121332 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.188863357000001, 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:13.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:13.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:13.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:13.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:13.538 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:13.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:13.541 INFO:teuthology.orchestra.run.smithi031.stdout:2349 write 0xcf6238a thru 0xcf66c34 (0x48ab bytes) 2024-09-13T09:08:13.541 INFO:teuthology.orchestra.run.smithi031.stdout:2350 write 0x828dca3 thru 0x829afd7 (0xd335 bytes) 2024-09-13T09:08:13.541 INFO:teuthology.orchestra.run.smithi031.stdout:2351 punch from 0xd05b21 to 0xd0f1ab, (0x968a bytes) 2024-09-13T09:08:13.541 INFO:teuthology.orchestra.run.smithi031.stdout:2352 write 0xa89bc52 thru 0xa8a49a5 (0x8d54 bytes) 2024-09-13T09:08:13.541 INFO:teuthology.orchestra.run.smithi031.stdout:2353 read 0x2f04493 thru 0x2f122ed (0xde5b bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2354 write 0x947fc62 thru 0x9481fbc (0x235b bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2355 read 0x804c753 thru 0x8051a4b (0x52f9 bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2356 read 0x6d655de thru 0x6d6bbd9 (0x65fc bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2357 writesame 0x37a3d36 thru 0x37a8649 data_size 0x6(0x4914 bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2358 write 0x33aecc7 thru 0x33bcbed (0xdf27 bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2359 write 0x1fefcbc thru 0x1ff5911 (0x5c56 bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2360 write 0x9003f02 thru 0x90069aa (0x2aa9 bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2361 writesame 0x29bbd37 thru 0x29c4da2 data_size 0xd(0x906c bytes) 2024-09-13T09:08:13.542 INFO:teuthology.orchestra.run.smithi031.stdout:2362 trunc from 0xcf66c35 to 0x2bce244 2024-09-13T09:08:13.542 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:13.535+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:13.553 INFO:teuthology.orchestra.run.smithi031.stdout:2364 write 0x39926eb thru 0x399e2e1 (0xbbf7 bytes) 2024-09-13T09:08:13.558 INFO:teuthology.orchestra.run.smithi031.stdout:2365 punch from 0x38f303e to 0x38f6783, (0x3745 bytes) 2024-09-13T09:08:13.558 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.568 INFO:teuthology.orchestra.run.smithi031.stdout:2367 trunc from 0x399e2e2 to 0x9a08447 2024-09-13T09:08:13.575 INFO:teuthology.orchestra.run.smithi031.stdout:2368 write 0x4cd2bc6 thru 0x4cdf28b (0xc6c6 bytes) 2024-09-13T09:08:13.578 INFO:teuthology.orchestra.run.smithi031.stdout:2369 write 0xa0de14 thru 0xa1d6bc (0xf8a9 bytes) 2024-09-13T09:08:13.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:08:13.581 INFO:teuthology.orchestra.run.smithi031.stdout:2370 read 0x553c3b8 thru 0x55420de (0x5d27 bytes) 2024-09-13T09:08:13.582 INFO:teuthology.orchestra.run.smithi031.stdout:2371 writesame 0x8caedce thru 0x8cb6209 data_size 0x56(0x743c bytes) 2024-09-13T09:08:13.584 INFO:teuthology.orchestra.run.smithi031.stdout:2372 punch from 0x335231f to 0x335bd2e, (0x9a0f bytes) 2024-09-13T09:08:13.584 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.584 INFO:teuthology.orchestra.run.smithi031.stdout:2375 write 0x6aa6350 thru 0x6ab205d (0xbd0e bytes) 2024-09-13T09:08:13.589 INFO:teuthology.orchestra.run.smithi031.stdout:2376 write 0x8b13ec8 thru 0x8b19fab (0x60e4 bytes) 2024-09-13T09:08:13.591 INFO:teuthology.orchestra.run.smithi031.stdout:2377 read 0x3ec8aaf thru 0x3ecc3c5 (0x3917 bytes) 2024-09-13T09:08:13.591 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.591 INFO:teuthology.orchestra.run.smithi031.stdout:2379 writesame 0x78ced75 thru 0x78d22ed data_size 0x27(0x3579 bytes) 2024-09-13T09:08:13.594 INFO:teuthology.orchestra.run.smithi031.stdout:2380 write 0x691636 thru 0x6a1383 (0xfd4e bytes) 2024-09-13T09:08:13.597 INFO:teuthology.orchestra.run.smithi031.stdout:2381 writesame 0x4a14941 thru 0x4a14f41 data_size 0x1(0x601 bytes) 2024-09-13T09:08:13.600 INFO:teuthology.orchestra.run.smithi031.stdout:2382 write 0x65c5e33 thru 0x65cb227 (0x53f5 bytes) 2024-09-13T09:08:13.604 INFO:teuthology.orchestra.run.smithi031.stdout:2384 write 0x189beef thru 0x18a2dec (0x6efe bytes) 2024-09-13T09:08:13.606 INFO:teuthology.orchestra.run.smithi031.stdout:2385 read 0x640fdac thru 0x641f431 (0xf686 bytes) 2024-09-13T09:08:13.607 INFO:teuthology.orchestra.run.smithi031.stdout:2386 read 0x35e8eed thru 0x35edd38 (0x4e4c bytes) 2024-09-13T09:08:13.607 INFO:teuthology.orchestra.run.smithi031.stdout:2387 punch from 0x796eb76 to 0x7973143, (0x45cd bytes) 2024-09-13T09:08:13.607 INFO:teuthology.orchestra.run.smithi031.stdout:2388 read 0x716c239 thru 0x7176bf3 (0xa9bb bytes) 2024-09-13T09:08:13.608 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.608 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.608 INFO:teuthology.orchestra.run.smithi031.stdout:2397 read 0x3ed1bd4 thru 0x3ed7751 (0x5b7e bytes) 2024-09-13T09:08:13.608 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.608 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.608 INFO:teuthology.orchestra.run.smithi031.stdout:2401 write 0x6a54281 thru 0x6a637e7 (0xf567 bytes) 2024-09-13T09:08:13.610 INFO:teuthology.orchestra.run.smithi031.stdout:2402 read 0x5355606 thru 0x535fde3 (0xa7de bytes) 2024-09-13T09:08:13.610 INFO:teuthology.orchestra.run.smithi031.stdout:2403 read 0x5345650 thru 0x534d0e4 (0x7a95 bytes) 2024-09-13T09:08:13.610 INFO:teuthology.orchestra.run.smithi031.stdout:2404 read 0x7d7fcd7 thru 0x7d873fd (0x7727 bytes) 2024-09-13T09:08:13.611 INFO:teuthology.orchestra.run.smithi031.stdout:2405 read 0x3c0e713 thru 0x3c0ed9b (0x689 bytes) 2024-09-13T09:08:13.616 INFO:teuthology.orchestra.run.smithi031.stdout:2407 trunc from 0x9a08447 to 0xc3c4141 2024-09-13T09:08:13.623 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.623 INFO:teuthology.orchestra.run.smithi031.stdout:2410 write 0x9d18e0e thru 0x9d2576e (0xc961 bytes) 2024-09-13T09:08:13.625 INFO:teuthology.orchestra.run.smithi031.stdout:2411 write 0x49646b1 thru 0x4969f0d (0x585d bytes) 2024-09-13T09:08:13.627 INFO:teuthology.orchestra.run.smithi031.stdout:2413 read 0xbd23511 thru 0xbd28ffb (0x5aeb bytes) 2024-09-13T09:08:13.627 INFO:teuthology.orchestra.run.smithi031.stdout:2414 read 0x394bf3a thru 0x395519e (0x9265 bytes) 2024-09-13T09:08:13.628 INFO:teuthology.orchestra.run.smithi031.stdout:2417 punch from 0xbe16365 to 0xbe256c4, (0xf35f bytes) 2024-09-13T09:08:13.629 INFO:teuthology.orchestra.run.smithi031.stdout:2418 write 0x1a1555b thru 0x1a1c89e (0x7344 bytes) 2024-09-13T09:08:13.631 INFO:teuthology.orchestra.run.smithi031.stdout:2419 punch from 0x843bc75 to 0x844abf9, (0xef84 bytes) 2024-09-13T09:08:13.631 INFO:teuthology.orchestra.run.smithi031.stdout:2420 read 0x7c5664 thru 0x7c83f0 (0x2d8d bytes) 2024-09-13T09:08:13.631 INFO:teuthology.orchestra.run.smithi031.stdout:2421 write 0x5b2a32f thru 0x5b2f4d4 (0x51a6 bytes) 2024-09-13T09:08:13.635 INFO:teuthology.orchestra.run.smithi031.stdout:2422 read 0xbdb4b0a thru 0xbdb7872 (0x2d69 bytes) 2024-09-13T09:08:13.635 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.639 INFO:teuthology.orchestra.run.smithi031.stdout:2425 trunc from 0xc3c4141 to 0xa5fbfe9 2024-09-13T09:08:13.640 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:13.633+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:13.651 INFO:teuthology.orchestra.run.smithi031.stdout:2426 write 0x4f68eac thru 0x4f6f367 (0x64bc bytes) 2024-09-13T09:08:13.654 INFO:teuthology.orchestra.run.smithi031.stdout:2427 punch from 0x6cd1a71 to 0x6cd46e7, (0x2c76 bytes) 2024-09-13T09:08:13.669 INFO:teuthology.orchestra.run.smithi031.stdout:2428 write 0xd69292d thru 0xd69e8c0 (0xbf94 bytes) 2024-09-13T09:08:13.671 INFO:teuthology.orchestra.run.smithi031.stdout:2429 write 0x8ee37c7 thru 0x8ee7752 (0x3f8c bytes) 2024-09-13T09:08:13.674 INFO:teuthology.orchestra.run.smithi031.stdout:2430 write 0xafcf9b8 thru 0xafd6767 (0x6db0 bytes) 2024-09-13T09:08:13.677 INFO:teuthology.orchestra.run.smithi031.stdout:2431 read 0x5b8a46e thru 0x5b97f91 (0xdb24 bytes) 2024-09-13T09:08:13.677 INFO:teuthology.orchestra.run.smithi031.stdout:2433 read 0x4fb7f56 thru 0x4fbd09a (0x5145 bytes) 2024-09-13T09:08:13.678 INFO:teuthology.orchestra.run.smithi031.stdout:2434 read 0x5ad5139 thru 0x5add515 (0x83dd bytes) 2024-09-13T09:08:13.679 INFO:teuthology.orchestra.run.smithi031.stdout:2435 read 0x7378357 thru 0x7382354 (0x9ffe bytes) 2024-09-13T09:08:13.682 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:13.682 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:13.694 INFO:teuthology.orchestra.run.smithi031.stdout:2436 trunc from 0xd69e8c1 to 0x412b050 2024-09-13T09:08:13.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:13.721 INFO:teuthology.orchestra.run.smithi031.stdout:2437 read 0x10ee682 thru 0x10f2478 (0x3df7 bytes) 2024-09-13T09:08:13.721 INFO:teuthology.orchestra.run.smithi031.stdout:2438 punch from 0x335a349 to 0x3367b2a, (0xd7e1 bytes) 2024-09-13T09:08:13.733 INFO:teuthology.orchestra.run.smithi031.stdout:2439 writesame 0x82b890a thru 0x82c169f data_size 0x15(0x8d96 bytes) 2024-09-13T09:08:13.735 INFO:teuthology.orchestra.run.smithi031.stdout:2440 read 0x50389bd thru 0x503ceff (0x4543 bytes) 2024-09-13T09:08:13.736 INFO:teuthology.orchestra.run.smithi031.stdout:2443 writesame 0x3e70bf thru 0x3f3ded data_size 0x1(0xcd2f bytes) 2024-09-13T09:08:13.741 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:13.735+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:13.778 INFO:teuthology.orchestra.run.smithi031.stdout:2444 trunc from 0x82c16a0 to 0x312f6d2 2024-09-13T09:08:13.793 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.793 INFO:teuthology.orchestra.run.smithi031.stdout:2446 read 0x263df2a thru 0x264855d (0xa634 bytes) 2024-09-13T09:08:13.793 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.793 INFO:teuthology.orchestra.run.smithi031.stdout:2449 writesame 0xbb5bfa thru 0xbbbee5 data_size 0x1a(0x62ec bytes) 2024-09-13T09:08:13.810 INFO:teuthology.orchestra.run.smithi031.stdout:2450 write 0x75d9211 thru 0x75e3ba0 (0xa990 bytes) 2024-09-13T09:08:13.812 INFO:teuthology.orchestra.run.smithi031.stdout:2451 punch from 0x14fb70 to 0x1516bc, (0x1b4c bytes) 2024-09-13T09:08:13.812 INFO:teuthology.orchestra.run.smithi031.stdout:2454 read 0x6d08e38 thru 0x6d0d40b (0x45d4 bytes) 2024-09-13T09:08:13.812 INFO:teuthology.orchestra.run.smithi031.stdout:2455 write 0x5101031 thru 0x5106a1e (0x59ee bytes) 2024-09-13T09:08:13.816 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:13.829 INFO:teuthology.orchestra.run.smithi031.stdout:2458 write 0xa5646a6 thru 0xa5710da (0xca35 bytes) 2024-09-13T09:08:13.831 INFO:teuthology.orchestra.run.smithi031.stdout:2459 write 0x8faa228 thru 0x8fb1275 (0x704e bytes) 2024-09-13T09:08:13.834 INFO:teuthology.orchestra.run.smithi031.stdout:2460 clone 23 order 22 su 131072 sc 4 2024-09-13T09:08:13.842 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:13.835+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:13.865 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:13.865 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:13.865 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:13.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:13.914 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:13.914 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:08:13.932 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-13T09:08:13.065863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":784983352,"num_objects":658,"num_object_clones":160,"num_object_copies":1316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":658,"num_whiteouts":171,"num_read":108379,"num_read_kb":381376,"num_write":7433,"num_write_kb":205325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":66610765,"num_keys_recovered":79,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5800,"ondisk_log_size":5800,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":536576,"kb_used_data":246508,"kb_used_omap":12,"kb_used_meta":289779,"kb_avail":754438144,"statfs":{"total":773094113280,"available":772544659456,"internally_reserved":0,"allocated":252424192,"data_stored":262028211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":296734264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10928650,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":2,"num_read":3728,"num_read_kb":17858,"num_write":285,"num_write_kb":1859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":18856692,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.214833"},"pg_stats":[{"pgid":"3.1f","version":"64'160","reported_seq":668,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.171013+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:08:08.171013+0000","last_peered":"2024-09-13T09:08:08.171013+0000","last_clean":"2024-09-13T09:08:08.171013+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:08:08.171013+0000","last_undegraded":"2024-09-13T09:08:08.171013+0000","last_fullsized":"2024-09-13T09:08:08.171013+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":10845646,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":843,"num_read_kb":4918,"num_write":213,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"64'208","reported_seq":1695,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.209774+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:08:08.209774+0000","last_peered":"2024-09-13T09:08:08.209774+0000","last_clean":"2024-09-13T09:08:08.209774+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:08:08.209774+0000","last_undegraded":"2024-09-13T09:08:08.209774+0000","last_fullsized":"2024-09-13T09:08:08.209774+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":37602897,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":2432,"num_read_kb":9910,"num_write":243,"num_write_kb":3848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"66'184","reported_seq":2322,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.496025+0000","last_change":"2024-09-13T09:07:15.879980+0000","last_active":"2024-09-13T09:08:10.496025+0000","last_peered":"2024-09-13T09:08:10.496025+0000","last_clean":"2024-09-13T09:08:10.496025+0000","last_became_active":"2024-09-13T09:07:13.856004+0000","last_became_peered":"2024-09-13T09:07:13.856004+0000","last_unstale":"2024-09-13T09:08:10.496025+0000","last_undegraded":"2024-09-13T09:08:10.496025+0000","last_fullsized":"2024-09-13T09:08:10.496025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:37.046343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":48416107,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":2126,"num_read_kb":25278,"num_write":258,"num_write_kb":13260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"67'95","reported_seq":1730,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.108640+0000","last_change":"2024-09-13T09:07:51.093764+0000","last_active":"2024-09-13T09:08:11.108640+0000","last_peered":"2024-09-13T09:08:11.108640+0000","last_clean":"2024-09-13T09:08:11.108640+0000","last_became_active":"2024-09-13T09:07:50.836531+0000","last_became_peered":"2024-09-13T09:07:50.836531+0000","last_unstale":"2024-09-13T09:08:11.108640+0000","last_undegraded":"2024-09-13T09:08:11.108640+0000","last_fullsized":"2024-09-13T09:08:11.108640+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:11.585290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":46102973,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2727,"num_read_kb":24227,"num_write":129,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":3944384,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"66'275","reported_seq":19516,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.281351+0000","last_change":"2024-09-13T09:07:16.108727+0000","last_active":"2024-09-13T09:08:10.281351+0000","last_peered":"2024-09-13T09:08:10.281351+0000","last_clean":"2024-09-13T09:08:10.281351+0000","last_became_active":"2024-09-13T09:07:13.862565+0000","last_became_peered":"2024-09-13T09:07:13.862565+0000","last_unstale":"2024-09-13T09:08:10.281351+0000","last_undegraded":"2024-09-13T09:08:10.281351+0000","last_fullsized":"2024-09-13T09:08:10.281351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:32.305423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":59106239,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":20552,"num_read_kb":38581,"num_write":348,"num_write_kb":16801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"67'209","reported_seq":2257,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.180099+0000","last_change":"2024-09-13T09:08:09.864519+0000","last_active":"2024-09-13T09:08:11.180099+0000","last_peered":"2024-09-13T09:08:11.180099+0000","last_clean":"2024-09-13T09:08:11.180099+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:08:11.180099+0000","last_undegraded":"2024-09-13T09:08:11.180099+0000","last_fullsized":"2024-09-13T09:08:11.180099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":47006338,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":3462,"num_read_kb":14124,"num_write":338,"num_write_kb":16596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"66'102","reported_seq":763,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.164299+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:08:10.164299+0000","last_peered":"2024-09-13T09:08:10.164299+0000","last_clean":"2024-09-13T09:08:10.164299+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:08:10.164299+0000","last_undegraded":"2024-09-13T09:08:10.164299+0000","last_fullsized":"2024-09-13T09:08:10.164299+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":2608235,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":580,"num_read_kb":3310,"num_write":102,"num_write_kb":4801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"59'154","reported_seq":1220,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976447+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:08:06.976447+0000","last_peered":"2024-09-13T09:08:06.976447+0000","last_clean":"2024-09-13T09:08:06.976447+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:08:06.976447+0000","last_undegraded":"2024-09-13T09:08:06.976447+0000","last_fullsized":"2024-09-13T09:08:06.976447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":18577308,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":1714,"num_read_kb":8005,"num_write":153,"num_write_kb":3690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"62'240","reported_seq":2822,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976621+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:08:06.976621+0000","last_peered":"2024-09-13T09:08:06.976621+0000","last_clean":"2024-09-13T09:08:06.976621+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:08:06.976621+0000","last_undegraded":"2024-09-13T09:08:06.976621+0000","last_fullsized":"2024-09-13T09:08:06.976621+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":42969160,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":3400,"num_read_kb":21858,"num_write":270,"num_write_kb":16106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"66'117","reported_seq":637,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.287631+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:08:10.287631+0000","last_peered":"2024-09-13T09:08:10.287631+0000","last_clean":"2024-09-13T09:08:10.287631+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:08:10.287631+0000","last_undegraded":"2024-09-13T09:08:10.287631+0000","last_fullsized":"2024-09-13T09:08:10.287631+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":8275651,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":642,"num_read_kb":3988,"num_write":133,"num_write_kb":3380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"67'214","reported_seq":2028,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.164568+0000","last_change":"2024-09-13T09:07:15.873331+0000","last_active":"2024-09-13T09:08:11.164568+0000","last_peered":"2024-09-13T09:08:11.164568+0000","last_clean":"2024-09-13T09:08:11.164568+0000","last_became_active":"2024-09-13T09:07:13.866813+0000","last_became_peered":"2024-09-13T09:07:13.866813+0000","last_unstale":"2024-09-13T09:08:11.164568+0000","last_undegraded":"2024-09-13T09:08:11.164568+0000","last_fullsized":"2024-09-13T09:08:11.164568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.439602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":45587035,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":2377,"num_read_kb":17801,"num_write":227,"num_write_kb":11713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"66'292","reported_seq":17270,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.102079+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:08:11.102079+0000","last_peered":"2024-09-13T09:08:11.102079+0000","last_clean":"2024-09-13T09:08:11.102079+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:08:11.102079+0000","last_undegraded":"2024-09-13T09:08:11.102079+0000","last_fullsized":"2024-09-13T09:08:11.102079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":16839736,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":5,"num_read":18705,"num_read_kb":25257,"num_write":266,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.907030+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:08:09.907030+0000","last_peered":"2024-09-13T09:08:09.907030+0000","last_clean":"2024-09-13T09:08:09.907030+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:08:09.907030+0000","last_undegraded":"2024-09-13T09:08:09.907030+0000","last_fullsized":"2024-09-13T09:08:09.907030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"66'359","reported_seq":1385,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.291042+0000","last_change":"2024-09-13T09:07:16.108420+0000","last_active":"2024-09-13T09:08:10.291042+0000","last_peered":"2024-09-13T09:08:10.291042+0000","last_clean":"2024-09-13T09:08:10.291042+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:08:10.291042+0000","last_undegraded":"2024-09-13T09:08:10.291042+0000","last_fullsized":"2024-09-13T09:08:10.291042+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":15010381,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":2414,"num_read_kb":8412,"num_write":462,"num_write_kb":1566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.859397+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:08:08.859397+0000","last_peered":"2024-09-13T09:08:08.859397+0000","last_clean":"2024-09-13T09:08:08.859397+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:08:08.859397+0000","last_undegraded":"2024-09-13T09:08:08.859397+0000","last_fullsized":"2024-09-13T09:08:08.859397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"66'85","reported_seq":986,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.281777+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:08:10.281777+0000","last_peered":"2024-09-13T09:08:10.281777+0000","last_clean":"2024-09-13T09:08:10.281777+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:08:10.281777+0000","last_undegraded":"2024-09-13T09:08:10.281777+0000","last_fullsized":"2024-09-13T09:08:10.281777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":12919040,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":905,"num_read_kb":10673,"num_write":86,"num_write_kb":3702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.907087+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:08:09.907087+0000","last_peered":"2024-09-13T09:08:09.907087+0000","last_clean":"2024-09-13T09:08:09.907087+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:08:09.907087+0000","last_undegraded":"2024-09-13T09:08:09.907087+0000","last_fullsized":"2024-09-13T09:08:09.907087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"66'262","reported_seq":687,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.928918+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:08:09.928918+0000","last_peered":"2024-09-13T09:08:09.928918+0000","last_clean":"2024-09-13T09:08:09.928918+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:08:09.928918+0000","last_undegraded":"2024-09-13T09:08:09.928918+0000","last_fullsized":"2024-09-13T09:08:09.928918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":22072637,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":1346,"num_read_kb":5247,"num_write":414,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976644+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:08:06.976644+0000","last_peered":"2024-09-13T09:08:06.976644+0000","last_clean":"2024-09-13T09:08:06.976644+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:08:06.976644+0000","last_undegraded":"2024-09-13T09:08:06.976644+0000","last_fullsized":"2024-09-13T09:08:06.976644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"66'193","reported_seq":2161,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.000201+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:08:11.000201+0000","last_peered":"2024-09-13T09:08:11.000201+0000","last_clean":"2024-09-13T09:08:11.000201+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:08:11.000201+0000","last_undegraded":"2024-09-13T09:08:11.000201+0000","last_fullsized":"2024-09-13T09:08:11.000201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":23677756,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":2593,"num_read_kb":14168,"num_write":295,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"67'117","reported_seq":1110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.035310+0000","last_change":"2024-09-13T09:07:15.873298+0000","last_active":"2024-09-13T09:08:11.035310+0000","last_peered":"2024-09-13T09:08:11.035310+0000","last_clean":"2024-09-13T09:08:11.035310+0000","last_became_active":"2024-09-13T09:07:13.852418+0000","last_became_peered":"2024-09-13T09:07:13.852418+0000","last_unstale":"2024-09-13T09:08:11.035310+0000","last_undegraded":"2024-09-13T09:08:11.035310+0000","last_fullsized":"2024-09-13T09:08:11.035310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:58.473601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":22868117,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":1554,"num_read_kb":7181,"num_write":127,"num_write_kb":977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":64,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.907124+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:08:09.907124+0000","last_peered":"2024-09-13T09:08:09.907124+0000","last_clean":"2024-09-13T09:08:09.907124+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:08:09.907124+0000","last_undegraded":"2024-09-13T09:08:09.907124+0000","last_fullsized":"2024-09-13T09:08:09.907124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"56'98","reported_seq":1538,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T09:08:06.976282+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:08:06.976282+0000","last_peered":"2024-09-13T09:08:06.976282+0000","last_clean":"2024-09-13T09:08:06.976282+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:08:06.976282+0000","last_undegraded":"2024-09-13T09:08:06.976282+0000","last_fullsized":"2024-09-13T09:08:06.976282+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":11014388,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":1444,"num_read_kb":10181,"num_write":100,"num_write_kb":4780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.853930+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:08:09.853930+0000","last_peered":"2024-09-13T09:08:09.853930+0000","last_clean":"2024-09-13T09:08:09.853930+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:08:09.853930+0000","last_undegraded":"2024-09-13T09:08:09.853930+0000","last_fullsized":"2024-09-13T09:08:09.853930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"66'117","reported_seq":941,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.929325+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:08:09.929325+0000","last_peered":"2024-09-13T09:08:09.929325+0000","last_clean":"2024-09-13T09:08:09.929325+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:08:09.929325+0000","last_undegraded":"2024-09-13T09:08:09.929325+0000","last_fullsized":"2024-09-13T09:08:09.929325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":5261862,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1023,"num_read_kb":6803,"num_write":110,"num_write_kb":3320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.906923+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:08:09.906923+0000","last_peered":"2024-09-13T09:08:09.906923+0000","last_clean":"2024-09-13T09:08:09.906923+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:08:09.906923+0000","last_undegraded":"2024-09-13T09:08:09.906923+0000","last_fullsized":"2024-09-13T09:08:09.906923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"66'156","reported_seq":380,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.427755+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:08:10.427755+0000","last_peered":"2024-09-13T09:08:10.427755+0000","last_clean":"2024-09-13T09:08:10.427755+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:08:10.427755+0000","last_undegraded":"2024-09-13T09:08:10.427755+0000","last_fullsized":"2024-09-13T09:08:10.427755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":1926788,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":594,"num_read_kb":3425,"num_write":200,"num_write_kb":1538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":63,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.907066+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:08:09.907066+0000","last_peered":"2024-09-13T09:08:09.907066+0000","last_clean":"2024-09-13T09:08:09.907066+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:08:09.907066+0000","last_undegraded":"2024-09-13T09:08:09.907066+0000","last_fullsized":"2024-09-13T09:08:09.907066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.855364+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:08:09.855364+0000","last_peered":"2024-09-13T09:08:09.855364+0000","last_clean":"2024-09-13T09:08:09.855364+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:08:09.855364+0000","last_undegraded":"2024-09-13T09:08:09.855364+0000","last_fullsized":"2024-09-13T09:08:09.855364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"66'166","reported_seq":2005,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.413917+0000","last_change":"2024-09-13T09:07:16.108021+0000","last_active":"2024-09-13T09:08:10.413917+0000","last_peered":"2024-09-13T09:08:10.413917+0000","last_clean":"2024-09-13T09:08:10.413917+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:08:10.413917+0000","last_undegraded":"2024-09-13T09:08:10.413917+0000","last_fullsized":"2024-09-13T09:08:10.413917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":26258161,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":2145,"num_read_kb":12140,"num_write":309,"num_write_kb":7485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"67'239","reported_seq":2105,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.146586+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:08:11.146586+0000","last_peered":"2024-09-13T09:08:11.146586+0000","last_clean":"2024-09-13T09:08:11.146586+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:08:11.146586+0000","last_undegraded":"2024-09-13T09:08:11.146586+0000","last_fullsized":"2024-09-13T09:08:11.146586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":33452997,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":3,"num_read":2959,"num_read_kb":10451,"num_write":316,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"67'159","reported_seq":718,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.001409+0000","last_change":"2024-09-13T09:07:53.864337+0000","last_active":"2024-09-13T09:08:11.001409+0000","last_peered":"2024-09-13T09:08:11.001409+0000","last_clean":"2024-09-13T09:08:11.001409+0000","last_became_active":"2024-09-13T09:07:53.862992+0000","last_became_peered":"2024-09-13T09:07:53.862992+0000","last_unstale":"2024-09-13T09:08:11.001409+0000","last_undegraded":"2024-09-13T09:08:11.001409+0000","last_fullsized":"2024-09-13T09:08:11.001409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:14.717209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":32169324,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":483,"num_read_kb":5893,"num_write":164,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":2632291,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"66'79","reported_seq":1283,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.494858+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:08:10.494858+0000","last_peered":"2024-09-13T09:08:10.494858+0000","last_clean":"2024-09-13T09:08:10.494858+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:08:10.494858+0000","last_undegraded":"2024-09-13T09:08:10.494858+0000","last_fullsized":"2024-09-13T09:08:10.494858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":8778164,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":1193,"num_read_kb":13116,"num_write":103,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"66'102","reported_seq":15684,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.035196+0000","last_change":"2024-09-13T09:07:15.873583+0000","last_active":"2024-09-13T09:08:11.035196+0000","last_peered":"2024-09-13T09:08:11.035196+0000","last_clean":"2024-09-13T09:08:11.035196+0000","last_became_active":"2024-09-13T09:07:13.859769+0000","last_became_peered":"2024-09-13T09:07:13.859769+0000","last_unstale":"2024-09-13T09:08:11.035196+0000","last_undegraded":"2024-09-13T09:08:11.035196+0000","last_fullsized":"2024-09-13T09:08:11.035196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:52.161129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":40982441,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":15563,"num_read_kb":23986,"num_write":151,"num_write_kb":12686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"66'73","reported_seq":1231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.501709+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:08:10.501709+0000","last_peered":"2024-09-13T09:08:10.501709+0000","last_clean":"2024-09-13T09:08:10.501709+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:08:10.501709+0000","last_undegraded":"2024-09-13T09:08:10.501709+0000","last_fullsized":"2024-09-13T09:08:10.501709+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":9877649,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1031,"num_read_kb":5678,"num_write":73,"num_write_kb":1430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"67'82","reported_seq":2162,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.215494+0000","last_change":"2024-09-13T09:07:15.859429+0000","last_active":"2024-09-13T09:08:11.215494+0000","last_peered":"2024-09-13T09:08:11.215494+0000","last_clean":"2024-09-13T09:08:11.215494+0000","last_became_active":"2024-09-13T09:07:13.855153+0000","last_became_peered":"2024-09-13T09:07:13.855153+0000","last_unstale":"2024-09-13T09:08:11.215494+0000","last_undegraded":"2024-09-13T09:08:11.215494+0000","last_fullsized":"2024-09-13T09:08:11.215494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:15.261153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":22940167,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2046,"num_read_kb":9681,"num_write":78,"num_write_kb":1763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"66'127","reported_seq":1749,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.498906+0000","last_change":"2024-09-13T09:07:51.052606+0000","last_active":"2024-09-13T09:08:10.498906+0000","last_peered":"2024-09-13T09:08:10.498906+0000","last_clean":"2024-09-13T09:08:10.498906+0000","last_became_active":"2024-09-13T09:07:50.848333+0000","last_became_peered":"2024-09-13T09:07:50.848333+0000","last_unstale":"2024-09-13T09:08:10.498906+0000","last_undegraded":"2024-09-13T09:08:10.498906+0000","last_fullsized":"2024-09-13T09:08:10.498906+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:23:05.120217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":14470592,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1913,"num_read_kb":5110,"num_write":126,"num_write_kb":1578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":885530,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"67'174","reported_seq":1656,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.208707+0000","last_change":"2024-09-13T09:08:09.863895+0000","last_active":"2024-09-13T09:08:11.208707+0000","last_peered":"2024-09-13T09:08:11.208707+0000","last_clean":"2024-09-13T09:08:11.208707+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:08:11.208707+0000","last_undegraded":"2024-09-13T09:08:11.208707+0000","last_fullsized":"2024-09-13T09:08:11.208707+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":45740110,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":1448,"num_read_kb":13900,"num_write":188,"num_write_kb":7774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"66'436","reported_seq":1782,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.233199+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:08:11.233199+0000","last_peered":"2024-09-13T09:08:11.233199+0000","last_clean":"2024-09-13T09:08:11.233199+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:08:11.233199+0000","last_undegraded":"2024-09-13T09:08:11.233199+0000","last_fullsized":"2024-09-13T09:08:11.233199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":15019178,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":3643,"num_read_kb":5104,"num_write":707,"num_write_kb":3417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"64'208","reported_seq":1956,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.853797+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:08:08.853797+0000","last_peered":"2024-09-13T09:08:08.853797+0000","last_clean":"2024-09-13T09:08:08.853797+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:08:08.853797+0000","last_undegraded":"2024-09-13T09:08:08.853797+0000","last_fullsized":"2024-09-13T09:08:08.853797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":31651121,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":2779,"num_read_kb":7288,"num_write":304,"num_write_kb":4178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"66'198","reported_seq":922,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.501819+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:08:10.501819+0000","last_peered":"2024-09-13T09:08:10.501819+0000","last_clean":"2024-09-13T09:08:10.501819+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:08:10.501819+0000","last_undegraded":"2024-09-13T09:08:10.501819+0000","last_fullsized":"2024-09-13T09:08:10.501819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":4364767,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":1627,"num_read_kb":5582,"num_write":279,"num_write_kb":4759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":784392965,"num_objects":654,"num_object_clones":160,"num_object_copies":1308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":654,"num_whiteouts":171,"num_read":108263,"num_read_kb":381276,"num_write":7272,"num_write_kb":202907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":66020397,"num_keys_recovered":79,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":231723008,"data_stored":244687266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5680,"ondisk_log_size":5680,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60312,"kb_used_data":25064,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94311528,"statfs":{"total":96636764160,"available":96575004672,"internally_reserved":0,"allocated":25665536,"data_stored":26003749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.873}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.1060000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.083}]}]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71756,"kb_used_data":36504,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94300084,"statfs":{"total":96636764160,"available":96563286016,"internally_reserved":0,"allocated":37380096,"data_stored":40358801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.314},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.29399999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.87}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.72599999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53588,"kb_used_data":18292,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94318252,"statfs":{"total":96636764160,"available":96581890048,"internally_reserved":0,"allocated":18731008,"data_stored":18897857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.2450000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.95299999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.228},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.016}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.042},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.075}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.1080000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.139},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.0620000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61712,"kb_used_data":30596,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94310128,"statfs":{"total":96636764160,"available":96573571072,"internally_reserved":0,"allocated":31330304,"data_stored":30705212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.255},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.27800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.753}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.69799999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79408,"kb_used_data":44176,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94292432,"statfs":{"total":96636764160,"available":96555450368,"internally_reserved":0,"allocated":45236224,"data_stored":50226741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.37}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.80900000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":20748,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":21245952,"data_stored":24284686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.93400000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.754},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.745}]},{"osd":3,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.83999999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.377}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.57599999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73300,"kb_used_data":33988,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94298540,"statfs":{"total":96636764160,"available":96561704960,"internally_reserved":0,"allocated":34803712,"data_stored":34159314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.82799999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.115},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.89200000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":1.244},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.4470000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.167},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.4019999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.613},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.544}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76444,"kb_used_data":37140,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94295396,"statfs":{"total":96636764160,"available":96558485504,"internally_reserved":0,"allocated":38031360,"data_stored":37391851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.501},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.9740000000000002,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.9050000000000002,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":8.0850000000000009,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.93100000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44478464,"data_stored":49886385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29974528,"data_stored":29774449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17977344,"data_stored":18557553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36618240,"data_stored":40018403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17817600,"data_stored":17624505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":39424000,"data_stored":39189561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20541440,"data_stored":23979167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24891392,"data_stored":25657243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:08:13.936 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:08:13.936 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:08:13.936 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:08:13.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:08:13.975 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:13.971+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:14.006 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:14.006 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9789535889999996, 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.007 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:08:14.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9789062560000001, 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.012 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9788487200000002, 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:08:14.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:08:14.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.978797234, 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:08:14.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:08:14.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:14.025 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.978750292, 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:14.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:08:14.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.978702899, 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.030 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:08:14.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.034 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9786556239999999, 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:14.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:08:14.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9786094240000001, 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9785454189999996, 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:08:14.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:08:14.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9760743300000003, 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:08:14.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:08:14.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9760186080000004, 2024-09-13T09:08:14.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:08:14.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9759687049999997, 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.058 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:14.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:08:14.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.975921649, 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.063 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9758731659999995, 2024-09-13T09:08:14.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.071 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9758079009999996, 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:14.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:08:14.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:08:14.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.975760985, 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:08:14.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9756735370000005, 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:08:14.080 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.084 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9742116880000005, 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.088 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9741336020000002, 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:08:14.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:08:14.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:08:14.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.9740815129999998, 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.098 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.098 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:14.099 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:14.100 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:14.095+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:14.117 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone22 intact 2024-09-13T09:08:14.200 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:14.195+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:14.232 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:14.232 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:14.232 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:14.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:14.251 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-13T09:08:14.287 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 69, 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T09:06:44.149006+0000", 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T09:08:14.013782+0000", 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T09:06:55.611487+0000", 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T09:07:49.811063+0000", 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-13T09:08:14.288 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T09:08:14.289 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:06:55.131330+0000", 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:14.290 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T09:08:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:14.293 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.294 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T09:08:14.295 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:14.296 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:02.743102+0000", 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:14.297 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:14.298 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-09-13T09:08:14.299 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:14.300 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:14.301 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9800000190734863, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9800000190734863, 2024-09-13T09:08:14.302 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 4.380000114440918, 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 4.380000114440918, 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:12.517158+0000", 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:08:14.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T09:08:14.304 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "69", 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:14.305 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 26, 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 69, 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:14.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:14.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:14.308 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T09:08:14.309 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-13T09:08:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-13T09:08:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-09-13T09:08:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:14.310 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "33c6fc7d-9828-4500-b103-b96d1cd7f420", 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 49, 2024-09-13T09:08:14.311 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:14.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T09:08:14.313 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6818/3291756747", 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6822/3291756747", 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6830/3291756747", 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6826/3291756747", 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:14.316 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "869ea3c5-8e6e-4b49-b70e-083f242a7aae", 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:14.317 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 65, 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T09:08:14.318 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T09:08:14.319 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/533520566", 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/533520566", 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/533520566", 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/533520566", 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:14.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e144d496-76ee-46e0-bdc6-135c77128535", 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 51, 2024-09-13T09:08:14.323 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:14.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:14.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T09:08:14.326 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.327 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6819/2253604482", 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6823/2253604482", 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/2253604482", 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6827/2253604482", 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "696b9414-9c19-4984-a8ff-14548e46b357", 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 52, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:14.329 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.332 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:14.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/3367214693", 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/3367214693", 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/3367214693", 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/3367214693", 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.334 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ac58040d-9b54-42ca-b143-1816abb54816", 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 49, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:14.335 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6800", 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6801", 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.336 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6802", 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6803", 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6806", 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6807", 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6804", 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6805", 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6801/3778074750", 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6803/3778074750", 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6807/3778074750", 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6805/3778074750", 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2", 2024-09-13T09:08:14.340 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 52, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.341 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6808", 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6809", 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6811", 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6813", 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6819", 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6824", 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6815", 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6817", 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6809/1926603087", 2024-09-13T09:08:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6813/1926603087", 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6824/1926603087", 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6817/1926603087", 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d", 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6820", 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6821", 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6823", 2024-09-13T09:08:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6825", 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6830", 2024-09-13T09:08:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6831", 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6827", 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6828", 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6821/766305500", 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6825/766305500", 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6831/766305500", 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6828/766305500", 2024-09-13T09:08:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f36c332c-9497-47e8-b943-f6b6dcaebeb8", 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 22, 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6810", 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6812", 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6814", 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6816", 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6826", 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6829", 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6818", 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6822", 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6812/1032956039", 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6816/1032956039", 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6829/1032956039", 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6822/1032956039", 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.183436+0000", 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.264080+0000", 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.253587+0000", 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.244429+0000", 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.369458+0000", 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.158089+0000", 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:08:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.217295+0000", 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.332254+0000", 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T09:08:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T09:08:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T09:08:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T09:08:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T09:08:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T09:08:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:08:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:08:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T09:08:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:14.366 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:14.368 INFO:tasks.thrashosds.thrasher:j is {'epoch': 69, 'fsid': 'fe16bf6d-4166-4417-80fd-1123617a33d3', 'created': '2024-09-13T09:06:44.149006+0000', 'modified': '2024-09-13T09:08:14.013782+0000', 'last_up_change': '2024-09-13T09:06:55.611487+0000', 'last_in_change': '2024-09-13T09:07:49.811063+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:06:55.131330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:07:02.743102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:07:12.517158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 69, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '33c6fc7d-9828-4500-b103-b96d1cd7f420', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6818', 'nonce': 3291756747}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6822', 'nonce': 3291756747}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6830', 'nonce': 3291756747}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6826', 'nonce': 3291756747}]}, 'public_addr': '172.21.15.31:6818/3291756747', 'cluster_addr': '172.21.15.31:6822/3291756747', 'heartbeat_back_addr': '172.21.15.31:6830/3291756747', 'heartbeat_front_addr': '172.21.15.31:6826/3291756747', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 533520566}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 533520566}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 533520566}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 533520566}]}, 'public_addr': '172.21.15.31:6801/533520566', 'cluster_addr': '172.21.15.31:6803/533520566', 'heartbeat_back_addr': '172.21.15.31:6807/533520566', 'heartbeat_front_addr': '172.21.15.31:6805/533520566', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e144d496-76ee-46e0-bdc6-135c77128535', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6817', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2253604482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6821', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2253604482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6829', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 2253604482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6825', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 2253604482}]}, 'public_addr': '172.21.15.31:6819/2253604482', 'cluster_addr': '172.21.15.31:6823/2253604482', 'heartbeat_back_addr': '172.21.15.31:6831/2253604482', 'heartbeat_front_addr': '172.21.15.31:6827/2253604482', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '696b9414-9c19-4984-a8ff-14548e46b357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 3367214693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 3367214693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 3367214693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 3367214693}]}, 'public_addr': '172.21.15.31:6809/3367214693', 'cluster_addr': '172.21.15.31:6811/3367214693', 'heartbeat_back_addr': '172.21.15.31:6815/3367214693', 'heartbeat_front_addr': '172.21.15.31:6813/3367214693', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ac58040d-9b54-42ca-b143-1816abb54816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6800', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6801', 'nonce': 3778074750}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 3778074750}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 3778074750}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 3778074750}]}, 'public_addr': '172.21.15.60:6801/3778074750', 'cluster_addr': '172.21.15.60:6803/3778074750', 'heartbeat_back_addr': '172.21.15.60:6807/3778074750', 'heartbeat_front_addr': '172.21.15.60:6805/3778074750', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1926603087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6811', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 1926603087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6819', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6824', 'nonce': 1926603087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6815', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 1926603087}]}, 'public_addr': '172.21.15.60:6809/1926603087', 'cluster_addr': '172.21.15.60:6813/1926603087', 'heartbeat_back_addr': '172.21.15.60:6824/1926603087', 'heartbeat_front_addr': '172.21.15.60:6817/1926603087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 766305500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6823', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 766305500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 766305500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6827', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6828', 'nonce': 766305500}]}, 'public_addr': '172.21.15.60:6821/766305500', 'cluster_addr': '172.21.15.60:6825/766305500', 'heartbeat_back_addr': '172.21.15.60:6831/766305500', 'heartbeat_front_addr': '172.21.15.60:6828/766305500', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f36c332c-9497-47e8-b943-f6b6dcaebeb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 22, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6812', 'nonce': 1032956039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6816', 'nonce': 1032956039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 1032956039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6822', 'nonce': 1032956039}]}, 'public_addr': '172.21.15.60:6812/1032956039', 'cluster_addr': '172.21.15.60:6816/1032956039', 'heartbeat_back_addr': '172.21.15.60:6829/1032956039', 'heartbeat_front_addr': '172.21.15.60:6822/1032956039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.183436+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.264080+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.253587+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.244429+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.369458+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.158089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.217295+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.332254+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:08:14.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:08:14.369 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:14.363+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:14.369 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:14.369 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:14.369 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:14.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:14.469 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:14.463+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.887776829000003, 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:08:14.508 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.887753498999999, 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:08:14.513 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:08:14.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.887730482000002, 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.518 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.887707352, 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.522 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:08:14.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.526 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.885090589000001, 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:14.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:08:14.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.885066502999997, 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.531 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:08:14.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.535 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.885039849999998, 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.536 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:14.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.885016210000003, 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:14.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:08:14.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.881200669000002, 2024-09-13T09:08:14.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:14.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:08:14.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.881177125999997, 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.550 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.553 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.554 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.881153836000003, 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.555 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:08:14.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.558 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.559 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.881130323000001, 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.564 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.881071867000003, 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.565 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:14.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:08:14.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.568 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.877040931000003, 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.569 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.570 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.573 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 21.557142634000002, 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:08:14.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:08:14.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 21.554770855000001, 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.579 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:08:14.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:08:14.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 21.554236939999999, 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:08:14.583 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:08:14.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 21.550836416999999, 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.589 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:08:14.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:14.593 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 21.54983661, 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:08:14.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 21.546666332000001, 2024-09-13T09:08:14.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:08:14.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:14.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:08:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:08:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:14.603 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:14.603 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:14.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:14.605 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:14.601+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:14.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:14.675 INFO:teuthology.orchestra.run.smithi031.stdout:2461 trunc from 0xa5710db to 0x402f54e 2024-09-13T09:08:14.692 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:14.704 INFO:teuthology.orchestra.run.smithi031.stdout:2463 writesame 0x8ee26bd thru 0x8ee6b11 data_size 0x77(0x4455 bytes) 2024-09-13T09:08:14.704 INFO:teuthology.orchestra.run.smithi031.stdout:2465 read 0x896e93b thru 0x89775e9 (0x8caf bytes) 2024-09-13T09:08:14.705 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:14.699+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:14.715 INFO:teuthology.orchestra.run.smithi031.stdout:2466 write 0xc69db22 thru 0xc6a0854 (0x2d33 bytes) 2024-09-13T09:08:14.719 INFO:teuthology.orchestra.run.smithi031.stdout:2467 write 0x741952d thru 0x7419cd8 (0x7ac bytes) 2024-09-13T09:08:14.722 INFO:teuthology.orchestra.run.smithi031.stdout:2468 write 0x6c1fdae thru 0x6c2deca (0xe11d bytes) 2024-09-13T09:08:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:14.726 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:08:14.727 INFO:teuthology.orchestra.run.smithi031.stdout:2469 read 0x9158c8 thru 0x9227d4 (0xcf0d bytes) 2024-09-13T09:08:14.742 INFO:teuthology.orchestra.run.smithi031.stdout:2473 trunc from 0xc6a0855 to 0x72f32 2024-09-13T09:08:14.755 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-13T09:08:14.022520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":787136995,"num_objects":673,"num_object_clones":161,"num_object_copies":1346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":673,"num_whiteouts":181,"num_read":112848,"num_read_kb":392631,"num_write":7842,"num_write_kb":205907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":392,"num_bytes_recovered":66610765,"num_keys_recovered":79,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6063,"ondisk_log_size":6063,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":541348,"kb_used_data":251168,"kb_used_omap":12,"kb_used_meta":289779,"kb_avail":754433372,"statfs":{"total":773094113280,"available":772539772928,"internally_reserved":0,"allocated":257196032,"data_stored":266732105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":296734264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13082293,"num_objects":40,"num_object_clones":1,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":8197,"num_read_kb":29113,"num_write":694,"num_write_kb":2441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":18856692,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.957542"},"pg_stats":[{"pgid":"3.1f","version":"67'293","reported_seq":3002,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:12.435511+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:08:12.435511+0000","last_peered":"2024-09-13T09:08:12.435511+0000","last_clean":"2024-09-13T09:08:12.435511+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:08:12.435511+0000","last_undegraded":"2024-09-13T09:08:12.435511+0000","last_fullsized":"2024-09-13T09:08:12.435511+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":11539324,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":3718,"num_read_kb":12049,"num_write":453,"num_write_kb":3218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"67'284","reported_seq":1780,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:12.835905+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:08:12.835905+0000","last_peered":"2024-09-13T09:08:12.835905+0000","last_clean":"2024-09-13T09:08:12.835905+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:08:12.835905+0000","last_undegraded":"2024-09-13T09:08:12.835905+0000","last_fullsized":"2024-09-13T09:08:12.835905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":39015742,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":2779,"num_read_kb":10187,"num_write":369,"num_write_kb":4063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"66'184","reported_seq":2322,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.496025+0000","last_change":"2024-09-13T09:07:15.879980+0000","last_active":"2024-09-13T09:08:10.496025+0000","last_peered":"2024-09-13T09:08:10.496025+0000","last_clean":"2024-09-13T09:08:10.496025+0000","last_became_active":"2024-09-13T09:07:13.856004+0000","last_became_peered":"2024-09-13T09:07:13.856004+0000","last_unstale":"2024-09-13T09:08:10.496025+0000","last_undegraded":"2024-09-13T09:08:10.496025+0000","last_fullsized":"2024-09-13T09:08:10.496025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:05:37.046343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":48416107,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":9,"num_read":2126,"num_read_kb":25278,"num_write":258,"num_write_kb":13260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"67'95","reported_seq":1730,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.108640+0000","last_change":"2024-09-13T09:07:51.093764+0000","last_active":"2024-09-13T09:08:11.108640+0000","last_peered":"2024-09-13T09:08:11.108640+0000","last_clean":"2024-09-13T09:08:11.108640+0000","last_became_active":"2024-09-13T09:07:50.836531+0000","last_became_peered":"2024-09-13T09:07:50.836531+0000","last_unstale":"2024-09-13T09:08:11.108640+0000","last_undegraded":"2024-09-13T09:08:11.108640+0000","last_fullsized":"2024-09-13T09:08:11.108640+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:11.585290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":46102973,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2727,"num_read_kb":24227,"num_write":129,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":3944384,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"66'275","reported_seq":19516,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.281351+0000","last_change":"2024-09-13T09:07:16.108727+0000","last_active":"2024-09-13T09:08:10.281351+0000","last_peered":"2024-09-13T09:08:10.281351+0000","last_clean":"2024-09-13T09:08:10.281351+0000","last_became_active":"2024-09-13T09:07:13.862565+0000","last_became_peered":"2024-09-13T09:07:13.862565+0000","last_unstale":"2024-09-13T09:08:10.281351+0000","last_undegraded":"2024-09-13T09:08:10.281351+0000","last_fullsized":"2024-09-13T09:08:10.281351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:31:32.305423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":59106239,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":20552,"num_read_kb":38581,"num_write":348,"num_write_kb":16801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"67'209","reported_seq":2257,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.180099+0000","last_change":"2024-09-13T09:08:09.864519+0000","last_active":"2024-09-13T09:08:11.180099+0000","last_peered":"2024-09-13T09:08:11.180099+0000","last_clean":"2024-09-13T09:08:11.180099+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:08:11.180099+0000","last_undegraded":"2024-09-13T09:08:11.180099+0000","last_fullsized":"2024-09-13T09:08:11.180099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":47006338,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":3462,"num_read_kb":14124,"num_write":338,"num_write_kb":16596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"66'102","reported_seq":763,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.164299+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:08:10.164299+0000","last_peered":"2024-09-13T09:08:10.164299+0000","last_clean":"2024-09-13T09:08:10.164299+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:08:10.164299+0000","last_undegraded":"2024-09-13T09:08:10.164299+0000","last_fullsized":"2024-09-13T09:08:10.164299+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":2608235,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":580,"num_read_kb":3310,"num_write":102,"num_write_kb":4801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"67'169","reported_seq":1248,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:12.737753+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:08:12.737753+0000","last_peered":"2024-09-13T09:08:12.737753+0000","last_clean":"2024-09-13T09:08:12.737753+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:08:12.737753+0000","last_undegraded":"2024-09-13T09:08:12.737753+0000","last_fullsized":"2024-09-13T09:08:12.737753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":18577324,"num_objects":24,"num_object_clones":5,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":1733,"num_read_kb":8012,"num_write":167,"num_write_kb":3760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"67'255","reported_seq":3890,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:12.364719+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:08:12.364719+0000","last_peered":"2024-09-13T09:08:12.364719+0000","last_clean":"2024-09-13T09:08:12.364719+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:08:12.364719+0000","last_undegraded":"2024-09-13T09:08:12.364719+0000","last_fullsized":"2024-09-13T09:08:12.364719+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":42960968,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":4599,"num_read_kb":25644,"num_write":281,"num_write_kb":16126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"66'117","reported_seq":637,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.287631+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:08:10.287631+0000","last_peered":"2024-09-13T09:08:10.287631+0000","last_clean":"2024-09-13T09:08:10.287631+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:08:10.287631+0000","last_undegraded":"2024-09-13T09:08:10.287631+0000","last_fullsized":"2024-09-13T09:08:10.287631+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":8275651,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":8,"num_read":642,"num_read_kb":3988,"num_write":133,"num_write_kb":3380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"67'214","reported_seq":2028,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.164568+0000","last_change":"2024-09-13T09:07:15.873331+0000","last_active":"2024-09-13T09:08:11.164568+0000","last_peered":"2024-09-13T09:08:11.164568+0000","last_clean":"2024-09-13T09:08:11.164568+0000","last_became_active":"2024-09-13T09:07:13.866813+0000","last_became_peered":"2024-09-13T09:07:13.866813+0000","last_unstale":"2024-09-13T09:08:11.164568+0000","last_undegraded":"2024-09-13T09:08:11.164568+0000","last_fullsized":"2024-09-13T09:08:11.164568+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:30:15.439602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":45587035,"num_objects":27,"num_object_clones":9,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":2377,"num_read_kb":17801,"num_write":227,"num_write_kb":11713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"66'292","reported_seq":17270,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.102079+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:08:11.102079+0000","last_peered":"2024-09-13T09:08:11.102079+0000","last_clean":"2024-09-13T09:08:11.102079+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:08:11.102079+0000","last_undegraded":"2024-09-13T09:08:11.102079+0000","last_fullsized":"2024-09-13T09:08:11.102079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":16839736,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":5,"num_read":18705,"num_read_kb":25257,"num_write":266,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.907030+0000","last_change":"2024-09-13T09:07:06.151546+0000","last_active":"2024-09-13T09:08:09.907030+0000","last_peered":"2024-09-13T09:08:09.907030+0000","last_clean":"2024-09-13T09:08:09.907030+0000","last_became_active":"2024-09-13T09:07:04.129822+0000","last_became_peered":"2024-09-13T09:07:04.129822+0000","last_unstale":"2024-09-13T09:08:09.907030+0000","last_undegraded":"2024-09-13T09:08:09.907030+0000","last_fullsized":"2024-09-13T09:08:09.907030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:43.573972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"66'359","reported_seq":1385,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.291042+0000","last_change":"2024-09-13T09:07:16.108420+0000","last_active":"2024-09-13T09:08:10.291042+0000","last_peered":"2024-09-13T09:08:10.291042+0000","last_clean":"2024-09-13T09:08:10.291042+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:08:10.291042+0000","last_undegraded":"2024-09-13T09:08:10.291042+0000","last_fullsized":"2024-09-13T09:08:10.291042+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":15010381,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":2,"num_read":2414,"num_read_kb":8412,"num_write":462,"num_write_kb":1566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":61,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.859397+0000","last_change":"2024-09-13T09:07:06.247823+0000","last_active":"2024-09-13T09:08:08.859397+0000","last_peered":"2024-09-13T09:08:08.859397+0000","last_clean":"2024-09-13T09:08:08.859397+0000","last_became_active":"2024-09-13T09:07:04.133177+0000","last_became_peered":"2024-09-13T09:07:04.133177+0000","last_unstale":"2024-09-13T09:08:08.859397+0000","last_undegraded":"2024-09-13T09:08:08.859397+0000","last_fullsized":"2024-09-13T09:08:08.859397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:20.894451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"66'85","reported_seq":986,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.281777+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:08:10.281777+0000","last_peered":"2024-09-13T09:08:10.281777+0000","last_clean":"2024-09-13T09:08:10.281777+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:08:10.281777+0000","last_undegraded":"2024-09-13T09:08:10.281777+0000","last_fullsized":"2024-09-13T09:08:10.281777+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":12919040,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":905,"num_read_kb":10673,"num_write":86,"num_write_kb":3702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.907087+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:08:09.907087+0000","last_peered":"2024-09-13T09:08:09.907087+0000","last_clean":"2024-09-13T09:08:09.907087+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:08:09.907087+0000","last_undegraded":"2024-09-13T09:08:09.907087+0000","last_fullsized":"2024-09-13T09:08:09.907087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"66'262","reported_seq":687,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.928918+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:08:09.928918+0000","last_peered":"2024-09-13T09:08:09.928918+0000","last_clean":"2024-09-13T09:08:09.928918+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:08:09.928918+0000","last_undegraded":"2024-09-13T09:08:09.928918+0000","last_fullsized":"2024-09-13T09:08:09.928918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":22072637,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":1346,"num_read_kb":5247,"num_write":414,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.010330+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:08:11.010330+0000","last_peered":"2024-09-13T09:08:11.010330+0000","last_clean":"2024-09-13T09:08:11.010330+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:08:11.010330+0000","last_undegraded":"2024-09-13T09:08:11.010330+0000","last_fullsized":"2024-09-13T09:08:11.010330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"66'193","reported_seq":2161,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.000201+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:08:11.000201+0000","last_peered":"2024-09-13T09:08:11.000201+0000","last_clean":"2024-09-13T09:08:11.000201+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:08:11.000201+0000","last_undegraded":"2024-09-13T09:08:11.000201+0000","last_fullsized":"2024-09-13T09:08:11.000201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":23677756,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":2593,"num_read_kb":14168,"num_write":295,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"67'117","reported_seq":1110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.035310+0000","last_change":"2024-09-13T09:07:15.873298+0000","last_active":"2024-09-13T09:08:11.035310+0000","last_peered":"2024-09-13T09:08:11.035310+0000","last_clean":"2024-09-13T09:08:11.035310+0000","last_became_active":"2024-09-13T09:07:13.852418+0000","last_became_peered":"2024-09-13T09:07:13.852418+0000","last_unstale":"2024-09-13T09:08:11.035310+0000","last_undegraded":"2024-09-13T09:08:11.035310+0000","last_fullsized":"2024-09-13T09:08:11.035310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:55:58.473601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":22868117,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":1554,"num_read_kb":7181,"num_write":127,"num_write_kb":977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":64,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.907124+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:08:09.907124+0000","last_peered":"2024-09-13T09:08:09.907124+0000","last_clean":"2024-09-13T09:08:09.907124+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:08:09.907124+0000","last_undegraded":"2024-09-13T09:08:09.907124+0000","last_fullsized":"2024-09-13T09:08:09.907124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"67'122","reported_seq":1583,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:12.329383+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:08:12.329383+0000","last_peered":"2024-09-13T09:08:12.329383+0000","last_clean":"2024-09-13T09:08:12.329383+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:08:12.329383+0000","last_undegraded":"2024-09-13T09:08:12.329383+0000","last_fullsized":"2024-09-13T09:08:12.329383+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":11069684,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":1473,"num_read_kb":10235,"num_write":118,"num_write_kb":4810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.853930+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:08:09.853930+0000","last_peered":"2024-09-13T09:08:09.853930+0000","last_clean":"2024-09-13T09:08:09.853930+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:08:09.853930+0000","last_undegraded":"2024-09-13T09:08:09.853930+0000","last_fullsized":"2024-09-13T09:08:09.853930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"66'117","reported_seq":941,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.929325+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:08:09.929325+0000","last_peered":"2024-09-13T09:08:09.929325+0000","last_clean":"2024-09-13T09:08:09.929325+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:08:09.929325+0000","last_undegraded":"2024-09-13T09:08:09.929325+0000","last_fullsized":"2024-09-13T09:08:09.929325+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":5261862,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":1023,"num_read_kb":6803,"num_write":110,"num_write_kb":3320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.906923+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:08:09.906923+0000","last_peered":"2024-09-13T09:08:09.906923+0000","last_clean":"2024-09-13T09:08:09.906923+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:08:09.906923+0000","last_undegraded":"2024-09-13T09:08:09.906923+0000","last_fullsized":"2024-09-13T09:08:09.906923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"66'156","reported_seq":380,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.427755+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:08:10.427755+0000","last_peered":"2024-09-13T09:08:10.427755+0000","last_clean":"2024-09-13T09:08:10.427755+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:08:10.427755+0000","last_undegraded":"2024-09-13T09:08:10.427755+0000","last_fullsized":"2024-09-13T09:08:10.427755+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":1926788,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":594,"num_read_kb":3425,"num_write":200,"num_write_kb":1538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":63,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.907066+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:08:09.907066+0000","last_peered":"2024-09-13T09:08:09.907066+0000","last_clean":"2024-09-13T09:08:09.907066+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:08:09.907066+0000","last_undegraded":"2024-09-13T09:08:09.907066+0000","last_fullsized":"2024-09-13T09:08:09.907066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":192,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:09.855364+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:08:09.855364+0000","last_peered":"2024-09-13T09:08:09.855364+0000","last_clean":"2024-09-13T09:08:09.855364+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:08:09.855364+0000","last_undegraded":"2024-09-13T09:08:09.855364+0000","last_fullsized":"2024-09-13T09:08:09.855364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"66'166","reported_seq":2005,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.413917+0000","last_change":"2024-09-13T09:07:16.108021+0000","last_active":"2024-09-13T09:08:10.413917+0000","last_peered":"2024-09-13T09:08:10.413917+0000","last_clean":"2024-09-13T09:08:10.413917+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:08:10.413917+0000","last_undegraded":"2024-09-13T09:08:10.413917+0000","last_fullsized":"2024-09-13T09:08:10.413917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":26258161,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":4,"num_read":2145,"num_read_kb":12140,"num_write":309,"num_write_kb":7485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"67'239","reported_seq":2105,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.146586+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:08:11.146586+0000","last_peered":"2024-09-13T09:08:11.146586+0000","last_clean":"2024-09-13T09:08:11.146586+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:08:11.146586+0000","last_undegraded":"2024-09-13T09:08:11.146586+0000","last_fullsized":"2024-09-13T09:08:11.146586+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":33452997,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":3,"num_read":2959,"num_read_kb":10451,"num_write":316,"num_write_kb":4676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"67'159","reported_seq":718,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.001409+0000","last_change":"2024-09-13T09:07:53.864337+0000","last_active":"2024-09-13T09:08:11.001409+0000","last_peered":"2024-09-13T09:08:11.001409+0000","last_clean":"2024-09-13T09:08:11.001409+0000","last_became_active":"2024-09-13T09:07:53.862992+0000","last_became_peered":"2024-09-13T09:07:53.862992+0000","last_unstale":"2024-09-13T09:08:11.001409+0000","last_undegraded":"2024-09-13T09:08:11.001409+0000","last_fullsized":"2024-09-13T09:08:11.001409+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:17:14.717209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":32169324,"num_objects":23,"num_object_clones":8,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":483,"num_read_kb":5893,"num_write":164,"num_write_kb":7597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":2632291,"num_keys_recovered":13,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"66'79","reported_seq":1283,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.494858+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:08:10.494858+0000","last_peered":"2024-09-13T09:08:10.494858+0000","last_clean":"2024-09-13T09:08:10.494858+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:08:10.494858+0000","last_undegraded":"2024-09-13T09:08:10.494858+0000","last_fullsized":"2024-09-13T09:08:10.494858+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":8778164,"num_objects":16,"num_object_clones":5,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":4,"num_read":1193,"num_read_kb":13116,"num_write":103,"num_write_kb":8076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"66'102","reported_seq":15684,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.035196+0000","last_change":"2024-09-13T09:07:15.873583+0000","last_active":"2024-09-13T09:08:11.035196+0000","last_peered":"2024-09-13T09:08:11.035196+0000","last_clean":"2024-09-13T09:08:11.035196+0000","last_became_active":"2024-09-13T09:07:13.859769+0000","last_became_peered":"2024-09-13T09:07:13.859769+0000","last_unstale":"2024-09-13T09:08:11.035196+0000","last_undegraded":"2024-09-13T09:08:11.035196+0000","last_fullsized":"2024-09-13T09:08:11.035196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:39:52.161129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":40982441,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":15563,"num_read_kb":23986,"num_write":151,"num_write_kb":12686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"66'73","reported_seq":1231,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.501709+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:08:10.501709+0000","last_peered":"2024-09-13T09:08:10.501709+0000","last_clean":"2024-09-13T09:08:10.501709+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:08:10.501709+0000","last_undegraded":"2024-09-13T09:08:10.501709+0000","last_fullsized":"2024-09-13T09:08:10.501709+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":9877649,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":1031,"num_read_kb":5678,"num_write":73,"num_write_kb":1430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"67'82","reported_seq":2162,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.215494+0000","last_change":"2024-09-13T09:07:15.859429+0000","last_active":"2024-09-13T09:08:11.215494+0000","last_peered":"2024-09-13T09:08:11.215494+0000","last_clean":"2024-09-13T09:08:11.215494+0000","last_became_active":"2024-09-13T09:07:13.855153+0000","last_became_peered":"2024-09-13T09:07:13.855153+0000","last_unstale":"2024-09-13T09:08:11.215494+0000","last_undegraded":"2024-09-13T09:08:11.215494+0000","last_fullsized":"2024-09-13T09:08:11.215494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:39:15.261153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":22940167,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2046,"num_read_kb":9681,"num_write":78,"num_write_kb":1763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"66'127","reported_seq":1749,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.498906+0000","last_change":"2024-09-13T09:07:51.052606+0000","last_active":"2024-09-13T09:08:10.498906+0000","last_peered":"2024-09-13T09:08:10.498906+0000","last_clean":"2024-09-13T09:08:10.498906+0000","last_became_active":"2024-09-13T09:07:50.848333+0000","last_became_peered":"2024-09-13T09:07:50.848333+0000","last_unstale":"2024-09-13T09:08:10.498906+0000","last_undegraded":"2024-09-13T09:08:10.498906+0000","last_fullsized":"2024-09-13T09:08:10.498906+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:23:05.120217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":14470592,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":5,"num_read":1913,"num_read_kb":5110,"num_write":126,"num_write_kb":1578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":885530,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"67'174","reported_seq":1656,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.208707+0000","last_change":"2024-09-13T09:08:09.863895+0000","last_active":"2024-09-13T09:08:11.208707+0000","last_peered":"2024-09-13T09:08:11.208707+0000","last_clean":"2024-09-13T09:08:11.208707+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:08:11.208707+0000","last_undegraded":"2024-09-13T09:08:11.208707+0000","last_fullsized":"2024-09-13T09:08:11.208707+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":45740110,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":1448,"num_read_kb":13900,"num_write":188,"num_write_kb":7774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"66'436","reported_seq":1782,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T09:08:11.233199+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:08:11.233199+0000","last_peered":"2024-09-13T09:08:11.233199+0000","last_clean":"2024-09-13T09:08:11.233199+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:08:11.233199+0000","last_undegraded":"2024-09-13T09:08:11.233199+0000","last_fullsized":"2024-09-13T09:08:11.233199+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":15019178,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":4,"num_read":3643,"num_read_kb":5104,"num_write":707,"num_write_kb":3417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"64'208","reported_seq":1956,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-13T09:08:08.853797+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:08:08.853797+0000","last_peered":"2024-09-13T09:08:08.853797+0000","last_clean":"2024-09-13T09:08:08.853797+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:08:08.853797+0000","last_undegraded":"2024-09-13T09:08:08.853797+0000","last_fullsized":"2024-09-13T09:08:08.853797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":31651121,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":2779,"num_read_kb":7288,"num_write":304,"num_write_kb":4178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"66'198","reported_seq":922,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-09-13T09:08:10.501819+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:08:10.501819+0000","last_peered":"2024-09-13T09:08:10.501819+0000","last_clean":"2024-09-13T09:08:10.501819+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:08:10.501819+0000","last_undegraded":"2024-09-13T09:08:10.501819+0000","last_fullsized":"2024-09-13T09:08:10.501819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":4364767,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":1627,"num_read_kb":5582,"num_write":279,"num_write_kb":4759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":786546608,"num_objects":669,"num_object_clones":161,"num_object_copies":1338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":669,"num_whiteouts":181,"num_read":112732,"num_read_kb":392531,"num_write":7681,"num_write_kb":203489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":390,"num_bytes_recovered":66020397,"num_keys_recovered":79,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233721856,"data_stored":246650465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5943,"ondisk_log_size":5943,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60312,"kb_used_data":25064,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94311528,"statfs":{"total":96636764160,"available":96575004672,"internally_reserved":0,"allocated":25665536,"data_stored":26003749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.96899999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.873}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.1060000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.083}]}]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71756,"kb_used_data":36504,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94300084,"statfs":{"total":96636764160,"available":96563286016,"internally_reserved":0,"allocated":37380096,"data_stored":40358801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.314},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.29399999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.36099999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.87}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.72599999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53588,"kb_used_data":18292,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94318252,"statfs":{"total":96636764160,"available":96581890048,"internally_reserved":0,"allocated":18731008,"data_stored":18897857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.2450000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.95299999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.228},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.016}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.042},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.075}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.1080000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":1.139},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.0620000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61712,"kb_used_data":30596,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94310128,"statfs":{"total":96636764160,"available":96573571072,"internally_reserved":0,"allocated":31330304,"data_stored":30705212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.255},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.27800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.47499999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.753}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.69799999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79408,"kb_used_data":44176,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94292432,"statfs":{"total":96636764160,"available":96555450368,"internally_reserved":0,"allocated":45236224,"data_stored":50226741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.37}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.86299999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.80900000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60056,"kb_used_data":20748,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94311784,"statfs":{"total":96636764160,"available":96575266816,"internally_reserved":0,"allocated":21245952,"data_stored":24284686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.93400000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.754},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.745}]},{"osd":3,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.83999999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.377}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.57599999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73300,"kb_used_data":33988,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94298540,"statfs":{"total":96636764160,"available":96561704960,"internally_reserved":0,"allocated":34803712,"data_stored":34159314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.82799999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.115},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.89200000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":1.244},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.4470000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.167},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.4019999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.613},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.544}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81216,"kb_used_data":41800,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94290624,"statfs":{"total":96636764160,"available":96553598976,"internally_reserved":0,"allocated":42803200,"data_stored":42095745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.9740000000000002,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.92200000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.79000000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.9050000000000002,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.748},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.0129999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.042},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":8.0850000000000009,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.0620000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.1040000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0860000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44478464,"data_stored":49886385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":29974528,"data_stored":29774449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17977344,"data_stored":18557553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":36618240,"data_stored":40018403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17817600,"data_stored":17624505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":41422848,"data_stored":41152760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20541440,"data_stored":23979167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24891392,"data_stored":25657243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:08:14.759 INFO:tasks.thrashosds.thrasher:Setting 3.a to [4, 2] 2024-09-13T09:08:14.759 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.a', '4', '2'] 2024-09-13T09:08:14.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.a 4 2 2024-09-13T09:08:14.784 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:14.785 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:14.791 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:14.792 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:14.792 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:14.799 INFO:teuthology.orchestra.run.smithi031.stdout:2475 punch from 0x56042 to 0x63364, (0xd322 bytes) 2024-09-13T09:08:14.802 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:14.802 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:14.805 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:14.799+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:14.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:14.814 INFO:teuthology.orchestra.run.smithi031.stdout:2476 write 0x62735e4 thru 0x627b835 (0x8252 bytes) 2024-09-13T09:08:14.817 INFO:teuthology.orchestra.run.smithi031.stdout:2477 read 0x18ced6 thru 0x194288 (0x73b3 bytes) 2024-09-13T09:08:14.820 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:14.820 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:14.835 INFO:teuthology.orchestra.run.smithi031.stdout:2480 writesame 0xdf09c99 thru 0xdf18058 data_size 0x20(0xe3c0 bytes) 2024-09-13T09:08:14.835 INFO:teuthology.orchestra.run.smithi031.stdout:2481 flatten 2024-09-13T09:08:14.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:14.837 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:14.841 INFO:teuthology.orchestra.run.smithi031.stdout:2482 punch from 0x11e2d81 to 0x11e68e9, (0x3b68 bytes) 2024-09-13T09:08:14.841 INFO:teuthology.orchestra.run.smithi031.stdout:2483 writesame 0x5d7906 thru 0x5e72cf data_size 0x1(0xf9ca bytes) 2024-09-13T09:08:14.854 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:14.854 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:14.871 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:14.872 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:14.889 INFO:teuthology.orchestra.run.smithi031.stdout:2484 write 0xd893100 thru 0xd897591 (0x4492 bytes) 2024-09-13T09:08:14.892 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:14.892 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:14.893 INFO:teuthology.orchestra.run.smithi031.stdout:2487 punch from 0xaf8ac12 to 0xaf918e0, (0x6cce bytes) 2024-09-13T09:08:14.893 INFO:teuthology.orchestra.run.smithi031.stdout:2488 writesame 0x2fcbc7d thru 0x2fd49d8 data_size 0x6d(0x8d5c bytes) 2024-09-13T09:08:14.897 INFO:teuthology.orchestra.run.smithi031.stdout:2489 write 0xd340370 thru 0xd3450e8 (0x4d79 bytes) 2024-09-13T09:08:14.900 INFO:teuthology.orchestra.run.smithi031.stdout:2490 write 0x4df8e88 thru 0x4e053a4 (0xc51d bytes) 2024-09-13T09:08:14.906 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:14.901+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:14.913 INFO:teuthology.orchestra.run.smithi031.stdout:2491 write 0xe607289 thru 0xe6132c9 (0xc041 bytes) 2024-09-13T09:08:14.916 INFO:teuthology.orchestra.run.smithi031.stdout:2492 write 0x5a8b3ab thru 0x5a96151 (0xada7 bytes) 2024-09-13T09:08:14.917 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:14.917 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:14.920 INFO:teuthology.orchestra.run.smithi031.stdout:2493 write 0x6d9f0f5 thru 0x6da29a8 (0x38b4 bytes) 2024-09-13T09:08:14.933 INFO:teuthology.orchestra.run.smithi031.stdout:2495 write 0xe773d46 thru 0xe780982 (0xcc3d bytes) 2024-09-13T09:08:14.936 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:14.936 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:14.937 INFO:teuthology.orchestra.run.smithi031.stdout:2496 writesame 0xb8b8d23 thru 0xb8bb180 data_size 0x13(0x245e bytes) 2024-09-13T09:08:14.941 INFO:teuthology.orchestra.run.smithi031.stdout:2497 read 0x118340c thru 0x11897f0 (0x63e5 bytes) 2024-09-13T09:08:14.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:14.952 INFO:teuthology.orchestra.run.smithi031.stdout:2499 trunc from 0xe780983 to 0x6cbf96f 2024-09-13T09:08:14.975 INFO:teuthology.orchestra.run.smithi031.stdout:2500 writesame 0x7a93a73 thru 0x7a9dd39 data_size 0x1(0xa2c7 bytes) 2024-09-13T09:08:14.975 INFO:teuthology.orchestra.run.smithi031.stdout:2502 write 0x5a2b5d3 thru 0x5a37c8d (0xc6bb bytes) 2024-09-13T09:08:14.978 INFO:teuthology.orchestra.run.smithi031.stdout:2503 punch from 0x6a2b912 to 0x6a2f142, (0x3830 bytes) 2024-09-13T09:08:14.979 INFO:teuthology.orchestra.run.smithi031.stdout:2504 read 0x454a3f2 thru 0x4550829 (0x6438 bytes) 2024-09-13T09:08:14.993 INFO:teuthology.orchestra.run.smithi031.stdout:2506 write 0xcf2088f thru 0xcf21611 (0xd83 bytes) 2024-09-13T09:08:14.995 INFO:teuthology.orchestra.run.smithi031.stdout:2508 punch from 0x402e86a to 0x4036739, (0x7ecf bytes) 2024-09-13T09:08:14.996 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:14.996 INFO:teuthology.orchestra.run.smithi031.stdout:2514 writesame 0x1f7003d thru 0x1f7351b data_size 0x1(0x34df bytes) 2024-09-13T09:08:15.004 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.004 INFO:teuthology.orchestra.run.smithi031.stdout:2517 punch from 0x355b78b to 0x355c404, (0xc79 bytes) 2024-09-13T09:08:15.014 INFO:teuthology.orchestra.run.smithi031.stdout:2519 writesame 0xe88f40c thru 0xe8980ec data_size 0x1(0x8ce1 bytes) 2024-09-13T09:08:15.014 INFO:teuthology.orchestra.run.smithi031.stdout:2520 write 0x8511d18 thru 0x851e71e (0xca07 bytes) 2024-09-13T09:08:15.017 INFO:teuthology.orchestra.run.smithi031.stdout:2522 read 0xe6486c3 thru 0xe65852e (0xfe6c bytes) 2024-09-13T09:08:15.017 INFO:teuthology.orchestra.run.smithi031.stdout:2523 write 0xb9cb035 thru 0xb9d8bd5 (0xdba1 bytes) 2024-09-13T09:08:15.021 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.021 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.021 INFO:teuthology.orchestra.run.smithi031.stdout:2526 read 0x2781845 thru 0x278bf57 (0xa713 bytes) 2024-09-13T09:08:15.021 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.029 INFO:teuthology.orchestra.run.smithi031.stdout:2529 trunc from 0xe8980ed to 0x8486c11 2024-09-13T09:08:15.036 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:15.033+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:15.047 INFO:teuthology.orchestra.run.smithi031.stdout:2532 punch from 0x329b81f to 0x329ea81, (0x3262 bytes) 2024-09-13T09:08:15.047 INFO:teuthology.orchestra.run.smithi031.stdout:2533 read 0x6382360 thru 0x6384b6f (0x2810 bytes) 2024-09-13T09:08:15.047 INFO:teuthology.orchestra.run.smithi031.stdout:2534 punch from 0x797ea29 to 0x798b072, (0xc649 bytes) 2024-09-13T09:08:15.047 INFO:teuthology.orchestra.run.smithi031.stdout:2535 write 0x69c8ebd thru 0x69d3eae (0xaff2 bytes) 2024-09-13T09:08:15.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:08:15.054 INFO:teuthology.orchestra.run.smithi031.stdout:2536 read 0x3484584 thru 0x348abd1 (0x664e bytes) 2024-09-13T09:08:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:2537 read 0x52ebe5a thru 0x52f752a (0xb6d1 bytes) 2024-09-13T09:08:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:2539 punch from 0x4b16aa to 0x4c1127, (0xfa7d bytes) 2024-09-13T09:08:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:2540 read 0x14197b5 thru 0x141d0c3 (0x390f bytes) 2024-09-13T09:08:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:2541 trunc from 0x8486c11 to 0xacf1d98 2024-09-13T09:08:15.067 INFO:teuthology.orchestra.run.smithi031.stdout:2542 trunc from 0xacf1d98 to 0xb0826a9 2024-09-13T09:08:15.074 INFO:teuthology.orchestra.run.smithi031.stdout:2543 writesame 0x5b47624 thru 0x5b50d27 data_size 0xa(0x9704 bytes) 2024-09-13T09:08:15.080 INFO:teuthology.orchestra.run.smithi031.stdout:2544 write 0x82324cc thru 0x823a1c8 (0x7cfd bytes) 2024-09-13T09:08:15.094 INFO:teuthology.orchestra.run.smithi031.stdout:2545 write 0xd0ebc54 thru 0xd0fa70a (0xeab7 bytes) 2024-09-13T09:08:15.101 INFO:teuthology.orchestra.run.smithi031.stdout:2546 trunc from 0xd0fa70b to 0xb05be74 2024-09-13T09:08:15.108 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:15.103+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:15.124 INFO:teuthology.orchestra.run.smithi031.stdout:2547 writesame 0xcac297c thru 0xcacf0b9 data_size 0x3(0xc73e bytes) 2024-09-13T09:08:15.124 INFO:teuthology.orchestra.run.smithi031.stdout:2548 read 0x7dbd16b thru 0x7dc6008 (0x8e9e bytes) 2024-09-13T09:08:15.124 INFO:teuthology.orchestra.run.smithi031.stdout:2549 write 0x2f063d2 thru 0x2f11409 (0xb038 bytes) 2024-09-13T09:08:15.128 INFO:teuthology.orchestra.run.smithi031.stdout:2550 punch from 0xa518299 to 0xa524ce4, (0xca4b bytes) 2024-09-13T09:08:15.128 INFO:teuthology.orchestra.run.smithi031.stdout:2552 read 0x2748bfc thru 0x274f038 (0x643d bytes) 2024-09-13T09:08:15.129 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:15.129 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:15.138 INFO:teuthology.orchestra.run.smithi031.stdout:2553 write 0xcf7add9 thru 0xcf87602 (0xc82a bytes) 2024-09-13T09:08:15.144 INFO:teuthology.orchestra.run.smithi031.stdout:2554 read 0xba95e7c thru 0xba9fd6a (0x9eef bytes) 2024-09-13T09:08:15.144 INFO:teuthology.orchestra.run.smithi031.stdout:2555 write 0x6dc6af0 thru 0x6dc9e1a (0x332b bytes) 2024-09-13T09:08:15.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:15.161 INFO:teuthology.orchestra.run.smithi031.stdout:2557 trunc from 0xcf87603 to 0x5ed5e38 2024-09-13T09:08:15.178 INFO:teuthology.orchestra.run.smithi031.stdout:2558 punch from 0x486455 to 0x492399, (0xbf44 bytes) 2024-09-13T09:08:15.179 INFO:teuthology.orchestra.run.smithi031.stdout:2559 punch from 0x573878e to 0x573e784, (0x5ff6 bytes) 2024-09-13T09:08:15.179 INFO:teuthology.orchestra.run.smithi031.stdout:2561 read 0xbcdf5f thru 0xbd787f (0x9921 bytes) 2024-09-13T09:08:15.179 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.179 INFO:teuthology.orchestra.run.smithi031.stdout:2563 read 0xcfb046 thru 0xd0a33b (0xf2f6 bytes) 2024-09-13T09:08:15.190 INFO:teuthology.orchestra.run.smithi031.stdout:2564 write 0x9b62020 thru 0x9b67916 (0x58f7 bytes) 2024-09-13T09:08:15.193 INFO:teuthology.orchestra.run.smithi031.stdout:2566 writesame 0x4952256 thru 0x495fa9a data_size 0x5(0xd845 bytes) 2024-09-13T09:08:15.202 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.211 INFO:teuthology.orchestra.run.smithi031.stdout:2568 write 0xb27f7de thru 0xb283f75 (0x4798 bytes) 2024-09-13T09:08:15.215 INFO:teuthology.orchestra.run.smithi031.stdout:2569 writesame 0x3c74ae thru 0x3d6730 data_size 0x31(0xf283 bytes) 2024-09-13T09:08:15.218 INFO:teuthology.orchestra.run.smithi031.stdout:2570 trunc from 0xb283f76 to 0xaae1a82 2024-09-13T09:08:15.236 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:15.233+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:2571 writesame 0xbd66d7a thru 0xbd6fdea data_size 0x1(0x9071 bytes) 2024-09-13T09:08:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:2574 read 0x2382589 thru 0x2385c5c (0x36d4 bytes) 2024-09-13T09:08:15.248 INFO:teuthology.orchestra.run.smithi031.stdout:2576 trunc from 0xbd6fdeb to 0x3068a07 2024-09-13T09:08:15.261 INFO:teuthology.orchestra.run.smithi031.stdout:2577 read 0xe1b37a thru 0xe21af8 (0x677f bytes) 2024-09-13T09:08:15.261 INFO:teuthology.orchestra.run.smithi031.stdout:2578 read 0x59d61 thru 0x65504 (0xb7a4 bytes) 2024-09-13T09:08:15.262 INFO:teuthology.orchestra.run.smithi031.stdout:2579 punch from 0x27f0603 to 0x27ffd52, (0xf74f bytes) 2024-09-13T09:08:15.262 INFO:teuthology.orchestra.run.smithi031.stdout:2580 write 0x15d6956 thru 0x15d78a2 (0xf4d bytes) 2024-09-13T09:08:15.273 INFO:teuthology.orchestra.run.smithi031.stdout:2582 trunc from 0x3068a07 to 0x81cb605 2024-09-13T09:08:15.280 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:15.280 INFO:teuthology.orchestra.run.smithi031.stdout:2585 read 0x44769f8 thru 0x4481c25 (0xb22e bytes) 2024-09-13T09:08:15.285 INFO:teuthology.orchestra.run.smithi031.stdout:2586 trunc from 0x81cb605 to 0x45897fa 2024-09-13T09:08:15.311 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:15.307+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:15.313 INFO:teuthology.orchestra.run.smithi031.stdout:2588 write 0x9e3f3b2 thru 0x9e4e808 (0xf457 bytes) 2024-09-13T09:08:15.315 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:15.315 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:15.319 INFO:teuthology.orchestra.run.smithi031.stdout:2590 writesame 0x4c4ddb7 thru 0x4c5d252 data_size 0x1f(0xf49c bytes) 2024-09-13T09:08:15.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:15.343 INFO:teuthology.orchestra.run.smithi031.stdout:2591 write 0xcd7a1b8 thru 0xcd8167c (0x74c5 bytes) 2024-09-13T09:08:15.367 INFO:teuthology.orchestra.run.smithi031.stdout:2592 trunc from 0xcd8167d to 0x4c76bf3 2024-09-13T09:08:15.389 INFO:teuthology.orchestra.run.smithi031.stdout:2594 read 0x2fb3eda thru 0x2fbe735 (0xa85c bytes) 2024-09-13T09:08:15.389 INFO:teuthology.orchestra.run.smithi031.stdout:2595 read 0x31554e0 thru 0x315d6f7 (0x8218 bytes) 2024-09-13T09:08:15.389 INFO:teuthology.orchestra.run.smithi031.stdout:2596 read 0x11a41c0 thru 0x11a8a51 (0x4892 bytes) 2024-09-13T09:08:15.389 INFO:teuthology.orchestra.run.smithi031.stdout:2598 punch from 0x4998ea to 0x4a30c0, (0x97d6 bytes) 2024-09-13T09:08:15.389 INFO:teuthology.orchestra.run.smithi031.stdout:2599 clone 24 order 24 su 2097152 sc 15 2024-09-13T09:08:15.417 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:15.413+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:15.501 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:15.501 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:15.512 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:15.509+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:15.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:15.612 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:15.609+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:15.684 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:15.684 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:15.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:15.741 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:15.737+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:15.815 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:15.809+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:15.831 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:15.831 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:15.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:15.916 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:15.909+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:15.965 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:15.965 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:15.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:16.016 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:16.013+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:16.033 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.a pg_upmap mapping to [4,2] 2024-09-13T09:08:16.033 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:08:16.106 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:16.106 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:16.117 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:16.111+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:16.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:16.218 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:16.211+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:16.273 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:16.273 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:16.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:16.327 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:16.323+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:16.420 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:16.415+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:16.451 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:16.451 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:16.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:16.555 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:16.551+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:16.621 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:16.622 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:16.622 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:16.617+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:16.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:16.722 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:16.721+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:16.798 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:16.798 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:16.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:16.823 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:16.821+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:16.924 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:16.917+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:16.935 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:16.935 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:16.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:16.949 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:17.035 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:17.031+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:17.076 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:17.076 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:17.091 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:17.091 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:17.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:17.107 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:17.115 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:17.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775741697000001, 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775724924000002, 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:08:17.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:08:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:08:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:08:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:08:17.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.125 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775687619000003, 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.126 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:08:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:17.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:08:17.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:08:17.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:08:17.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:08:17.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:08:17.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:08:17.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:08:17.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:08:17.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:08:17.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:08:17.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:08:17.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:08:17.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:08:17.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.136 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.137 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.137 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:17.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775667904999999, 2024-09-13T09:08:17.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:08:17.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:08:17.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:17.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:17.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775651580000002, 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:08:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:08:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775635714000003, 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:08:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:08:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775619929000001, 2024-09-13T09:08:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:08:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:08:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775600857999997, 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:08:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:08:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775584311000003, 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:08:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775556711999997, 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:08:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:08:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:08:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:08:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775539469999998, 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:08:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:08:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:08:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775509026999998, 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:08:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:08:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:08:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:08:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:08:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:08:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:08:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:08:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:08:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:08:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:08:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:08:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:08:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:08:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:08:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775490112, 2024-09-13T09:08:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:08:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:08:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:08:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:08:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:08:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:08:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:08:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:08:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:08:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:08:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:08:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:08:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:08:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:08:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:08:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:08:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775472323999999, 2024-09-13T09:08:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:08:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:08:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:08:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:08:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:08:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:08:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:08:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:08:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:08:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:08:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:08:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:08:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775447100000001, 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775428896000001, 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:08:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775408577999997, 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:08:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:08:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:08:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775390029999997, 2024-09-13T09:08:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:08:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775372756000003, 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:08:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:08:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:08:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.775355302000001, 2024-09-13T09:08:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:08:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:08:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.238 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:17.239 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:17.239 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:17.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:17.240 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:17.241 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:17.241 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:17.242 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:17.243 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:17.233+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:17.339 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:17.337+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:17.410 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:17.410 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:17.410 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:17.410 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:17.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:17.471 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:17.467+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:17.541 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:17.537+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:17.548 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:17.548 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:17.549 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:17.549 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:17.549 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:17.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:17.641 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:17.637+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:17.732 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:17.732 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:17.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:17.732 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:17.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:08:17.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.598271842000003, 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:08:17.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:08:17.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:08:17.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.769989662, 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.740 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:08:17.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:08:17.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56973 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4501504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.481084566, 2024-09-13T09:08:17.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091397946999999993, 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56973 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247918+0000", 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247921+0000", 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247927+0000", 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2389999999999999e-06 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247932+0000", 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339077+0000", 2024-09-13T09:08:17.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091144366000000004 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339203+0000", 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126919 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339317+0000", 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113205 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56975 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4515840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.480890799000001, 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091417921999999999, 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56975 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247936+0000", 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999824 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248117+0000", 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180827 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248127+0000", 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248133+0000", 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.19e-06 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339347+0000", 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091213581000000002 2024-09-13T09:08:17.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339419+0000", 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.203e-05 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339530+0000", 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111359 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56976 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3681280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.480847401999998, 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091677039000000002, 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56976 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.755 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248155+0000", 2024-09-13T09:08:17.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248159+0000", 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248166+0000", 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999999e-06 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248172+0000", 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339558+0000", 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091385818999999993 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339705+0000", 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014693599999999999 2024-09-13T09:08:17.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339833+0000", 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012808200000000001 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56990 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3795968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:17.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.477706900000001, 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091546566999999995, 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56990 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:17.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251298+0000", 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.041e-06 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251303+0000", 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251319+0000", 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5852e-05 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342591+0000", 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091271605000000006 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342728+0000", 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013662799999999999 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342843+0000", 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115442 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.763 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57022 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4334592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.477624273, 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091691988000000002, 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57022 2024-09-13T09:08:17.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251382+0000", 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5659999999999998e-06 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251391+0000", 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0170000000000002e-06 2024-09-13T09:08:17.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251397+0000", 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.908e-06 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342870+0000", 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091473410000000005 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342945+0000", 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4175e-05 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343071+0000", 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012638499999999999 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57026 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5113856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:17.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.477601335999999, 2024-09-13T09:08:17.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091880383999999996, 2024-09-13T09:08:17.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57026 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251404+0000", 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251410+0000", 2024-09-13T09:08:17.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251415+0000", 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2129999999999999e-06 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343098+0000", 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091683073000000004 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343171+0000", 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3149e-05 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343282+0000", 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110899 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57027 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4348928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.477583693, 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092079082000000007, 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57027 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251419+0000", 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251421+0000", 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251427+0000", 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4419999999999997e-06 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251432+0000", 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7620000000000003e-06 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343310+0000", 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091878572000000006 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343383+0000", 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2368999999999998e-05 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343499+0000", 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116142 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57028 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5128192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.477566348, 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092372671000000003, 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57028 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251439+0000", 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1730000000000002e-06 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251445+0000", 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251450+0000", 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343526+0000", 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092076621999999997 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343605+0000", 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8830000000000002e-05 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343810+0000", 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204544 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.783 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.477549824, 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:08:17.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:08:17.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:08:17.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:08:17.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:08:17.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.477470025999999, 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.789 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:08:17.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:08:17.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.477220955, 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:17.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:08:17.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.476907814, 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.799 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:08:17.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:08:17.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.476454581999999, 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:08:17.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.476430067999999, 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:08:17.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.476164838999999, 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:08:17.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:17.819 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.47587699, 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:17.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:08:17.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57037 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.475440789, 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092241045999999993, 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57037 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.825 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253152+0000", 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995894 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253567+0000", 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000414886 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253576+0000", 2024-09-13T09:08:17.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6750000000000008e-06 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253582+0000", 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345610+0000", 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092027943000000001 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345682+0000", 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2311999999999995e-05 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345804+0000", 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121332 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.475415686000002, 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.830 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:17.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:17.834 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:17.835 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:17.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:17.837 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:17.829+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:17.937 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:17.933+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:17.972 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:17.972 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:17.972 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:17.972 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:17.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:18.038 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:18.031+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:18.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:08:18.139 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:18.133+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:18.148 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:18.148 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:18.148 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:18.148 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:18.148 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:18.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:18.249 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:18.245+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.308055895000001, 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:08:18.334 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.338 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.308008561999999, 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:18.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:08:18.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.307951026, 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:18.347 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.307899539999999, 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:18.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:08:18.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.307852598, 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.352 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:08:18.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:08:18.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:18.356 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.307805204999999, 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:08:18.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.360 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.307757929999999, 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:08:18.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:08:18.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.307711729999999, 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.365 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:08:18.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.307647725000001, 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:08:18.369 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.373 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.305176636000001, 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:08:18.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:08:18.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.305120914, 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:08:18.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.305071011000001, 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:08:18.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.386 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.305023954999999, 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.387 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:08:18.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:08:18.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.304975472000001, 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:08:18.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.304910207000001, 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:08:18.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.304863291, 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.400 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:08:18.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:08:18.403 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.304775843, 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.404 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:08:18.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:08:18.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.303313994, 2024-09-13T09:08:18.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:08:18.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.303235908, 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.303183819000001, 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:08:18.417 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.421 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:18.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:18.423 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:18.417+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:18.524 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:18.517+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:18.596 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:18.597 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:18.597 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:18.597 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:18.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:18.625 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:18.619+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:18.726 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:18.719+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:18.780 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:18.780 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:18.780 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:18.780 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:18.780 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:18.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:18.827 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:18.819+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:18.928 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:18.928 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:18.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:18.928 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:18.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.305826689, 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:08:18.932 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.305803359000002, 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.933 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:08:18.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.937 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.305780341999998, 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.941 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.305757212000003, 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:08:18.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:08:18.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:08:18.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.946 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.303140448999997, 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:08:18.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:08:18.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.303116363000001, 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.303089710000002, 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.958 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:08:18.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.30306607, 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:08:18.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:08:18.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.299250528999998, 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:08:18.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.299226986000001, 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:08:18.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.977 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.977 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:18.977 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.299203695999999, 2024-09-13T09:08:18.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:08:18.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.977 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:18.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:08:18.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.299180182999997, 2024-09-13T09:08:18.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:08:18.986 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.299121726999999, 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.991 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 44.295090791, 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:18.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 25.975192494000002, 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:08:18.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:08:18.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:18.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 25.972820715000001, 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:08:19.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:08:19.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:08:19.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 25.972286799999999, 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:08:19.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:19.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:19.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 25.968886276999999, 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.012 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 25.96788647, 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:19.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:19.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:19.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.021 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 25.964716192000001, 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:08:19.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:08:19.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:08:19.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:19.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.027 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:19.027 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:19.027 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:19.027 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:18.921+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:19.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:19.068 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:08:19.073 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:19.069+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:19.113 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-13T09:08:19.131 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:19.125+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:19.218 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:19.219 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:19.219 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:19.219 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:19.231 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:19.225+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:19.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:19.243 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:19.331 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:19.325+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:19.393 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:19.393 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:19.405 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:19.405 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:19.405 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:19.405 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:19.405 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:19.411 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:19.411 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:19.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:19.428 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:19.428 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:19.445 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:19.461 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:19.457+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:19.463 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:19.463 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:19.464 INFO:teuthology.orchestra.run.smithi031.stdout:2600 write 0x8df5f8 thru 0x8e17f5 (0x21fe bytes) 2024-09-13T09:08:19.480 INFO:teuthology.orchestra.run.smithi031.stdout:2603 write 0x606145e thru 0x6061dc7 (0x96a bytes) 2024-09-13T09:08:19.485 INFO:teuthology.orchestra.run.smithi031.stdout:2605 read 0x8c88d thru 0x9ae32 (0xe5a6 bytes) 2024-09-13T09:08:19.486 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:19.486 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:19.498 INFO:teuthology.orchestra.run.smithi031.stdout:2606 write 0xa5cfc7a thru 0xa5dcb66 (0xceed bytes) 2024-09-13T09:08:19.502 INFO:teuthology.orchestra.run.smithi031.stdout:2607 read 0x68d122 thru 0x694259 (0x7138 bytes) 2024-09-13T09:08:19.506 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:19.507 INFO:teuthology.orchestra.run.smithi031.stdout:2608 trunc from 0xa5dcb67 to 0xcc91e6c 2024-09-13T09:08:19.512 INFO:teuthology.orchestra.run.smithi031.stdout:2609 read 0x114647d thru 0x114de54 (0x79d8 bytes) 2024-09-13T09:08:19.513 INFO:teuthology.orchestra.run.smithi031.stdout:2610 read 0x9158503 thru 0x915aa72 (0x2570 bytes) 2024-09-13T09:08:19.513 INFO:teuthology.orchestra.run.smithi031.stdout:2611 read 0x8646558 thru 0x8648b49 (0x25f2 bytes) 2024-09-13T09:08:19.519 INFO:teuthology.orchestra.run.smithi031.stdout:2612 trunc from 0xcc91e6c to 0x926d193 2024-09-13T09:08:19.528 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:19.528 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:19.534 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:19.527+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:19.553 INFO:teuthology.orchestra.run.smithi031.stdout:2613 punch from 0x58a8ae to 0x58d0d0, (0x2822 bytes) 2024-09-13T09:08:19.553 INFO:teuthology.orchestra.run.smithi031.stdout:2614 read 0x7d498fa thru 0x7d5397a (0xa081 bytes) 2024-09-13T09:08:19.553 INFO:teuthology.orchestra.run.smithi031.stdout:2615 read 0x7fe6120 thru 0x7fe7553 (0x1434 bytes) 2024-09-13T09:08:19.554 INFO:teuthology.orchestra.run.smithi031.stdout:2616 write 0xeb8b66 thru 0xeba5d2 (0x1a6d bytes) 2024-09-13T09:08:19.556 INFO:teuthology.orchestra.run.smithi031.stdout:2617 read 0x2b32690 thru 0x2b3bacc (0x943d bytes) 2024-09-13T09:08:19.556 INFO:teuthology.orchestra.run.smithi031.stdout:2618 read 0x3090f19 thru 0x3097c89 (0x6d71 bytes) 2024-09-13T09:08:19.557 INFO:teuthology.orchestra.run.smithi031.stdout:2624 read 0x7eb8434 thru 0x7ec5793 (0xd360 bytes) 2024-09-13T09:08:19.557 INFO:teuthology.orchestra.run.smithi031.stdout:2626 read 0x5f45b67 thru 0x5f4a2ae (0x4748 bytes) 2024-09-13T09:08:19.571 INFO:teuthology.orchestra.run.smithi031.stdout:2628 write 0xddba370 thru 0xddc11a9 (0x6e3a bytes) 2024-09-13T09:08:19.572 INFO:teuthology.orchestra.run.smithi031.stdout:2629 read 0x28eab8e thru 0x28faa44 (0xfeb7 bytes) 2024-09-13T09:08:19.573 INFO:teuthology.orchestra.run.smithi031.stdout:2630 write 0x481780c thru 0x4823eff (0xc6f4 bytes) 2024-09-13T09:08:19.574 INFO:teuthology.orchestra.run.smithi031.stdout:2632 read 0x4418742 thru 0x442183b (0x90fa bytes) 2024-09-13T09:08:19.574 INFO:teuthology.orchestra.run.smithi031.stdout:2633 write 0xd0d0fd7 thru 0xd0ddfcf (0xcff9 bytes) 2024-09-13T09:08:19.576 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.576 INFO:teuthology.orchestra.run.smithi031.stdout:2635 read 0x67267 thru 0x6f4f6 (0x8290 bytes) 2024-09-13T09:08:19.577 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.577 INFO:teuthology.orchestra.run.smithi031.stdout:2637 read 0x5e351b2 thru 0x5e3537c (0x1cb bytes) 2024-09-13T09:08:19.590 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:19.590 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:19.590 INFO:teuthology.orchestra.run.smithi031.stdout:2638 trunc from 0xddc11aa to 0x4df3366 2024-09-13T09:08:19.601 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.601 INFO:teuthology.orchestra.run.smithi031.stdout:2641 punch from 0x3691a3d to 0x3694974, (0x2f37 bytes) 2024-09-13T09:08:19.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:19.608 INFO:teuthology.orchestra.run.smithi031.stdout:2642 trunc from 0x4df3366 to 0xa20769d 2024-09-13T09:08:19.624 INFO:teuthology.orchestra.run.smithi031.stdout:2644 writesame 0xd720bb8 thru 0xd725aaf data_size 0x2(0x4ef8 bytes) 2024-09-13T09:08:19.626 INFO:teuthology.orchestra.run.smithi031.stdout:2645 writesame 0x66d49ef thru 0x66d7da0 data_size 0xd(0x33b2 bytes) 2024-09-13T09:08:19.629 INFO:teuthology.orchestra.run.smithi031.stdout:2646 write 0x95bca67 thru 0x95bd581 (0xb1b bytes) 2024-09-13T09:08:19.630 INFO:teuthology.orchestra.run.smithi031.stdout:2647 write 0x45d7902 thru 0x45dc22a (0x4929 bytes) 2024-09-13T09:08:19.631 INFO:teuthology.orchestra.run.smithi031.stdout:2649 read 0x7115207 thru 0x71181c2 (0x2fbc bytes) 2024-09-13T09:08:19.632 INFO:teuthology.orchestra.run.smithi031.stdout:2650 read 0x837ccbc thru 0x837f904 (0x2c49 bytes) 2024-09-13T09:08:19.632 INFO:teuthology.orchestra.run.smithi031.stdout:2651 write 0x1306e32 thru 0x130e706 (0x78d5 bytes) 2024-09-13T09:08:19.634 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.634 INFO:teuthology.orchestra.run.smithi031.stdout:2653 read 0x7b4313c thru 0x7b4d191 (0xa056 bytes) 2024-09-13T09:08:19.635 INFO:teuthology.orchestra.run.smithi031.stdout:2655 read 0xd0e24f7 thru 0xd0e5f44 (0x3a4e bytes) 2024-09-13T09:08:19.635 INFO:teuthology.orchestra.run.smithi031.stdout:2657 read 0xb2b20ec thru 0xb2bdf82 (0xbe97 bytes) 2024-09-13T09:08:19.635 INFO:teuthology.orchestra.run.smithi031.stdout:2660 write 0xb280e53 thru 0xb28eb6b (0xdd19 bytes) 2024-09-13T09:08:19.637 INFO:teuthology.orchestra.run.smithi031.stdout:2664 punch from 0x5e3fb6b to 0x5e41542, (0x19d7 bytes) 2024-09-13T09:08:19.637 INFO:teuthology.orchestra.run.smithi031.stdout:2665 read 0x6b53c6c thru 0x6b5c655 (0x89ea bytes) 2024-09-13T09:08:19.637 INFO:teuthology.orchestra.run.smithi031.stdout:2666 punch from 0xd5be6f7 to 0xd5c05f1, (0x1efa bytes) 2024-09-13T09:08:19.638 INFO:teuthology.orchestra.run.smithi031.stdout:2667 writesame 0x2cc557a thru 0x2cc9b5a data_size 0x43(0x45e1 bytes) 2024-09-13T09:08:19.639 INFO:teuthology.orchestra.run.smithi031.stdout:2671 read 0x6beb0ce thru 0x6bf8742 (0xd675 bytes) 2024-09-13T09:08:19.640 INFO:teuthology.orchestra.run.smithi031.stdout:2672 writesame 0xcc776e0 thru 0xcc7fd6c data_size 0x5(0x868d bytes) 2024-09-13T09:08:19.646 INFO:teuthology.orchestra.run.smithi031.stdout:2673 writesame 0x5673168 thru 0x567fb92 data_size 0xb(0xca2b bytes) 2024-09-13T09:08:19.651 INFO:teuthology.orchestra.run.smithi031.stdout:2674 read 0x7e74570 thru 0x7e7ae93 (0x6924 bytes) 2024-09-13T09:08:19.651 INFO:teuthology.orchestra.run.smithi031.stdout:2675 write 0x4eea675 thru 0x4ef1efb (0x7887 bytes) 2024-09-13T09:08:19.653 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:19.649+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:19.654 INFO:teuthology.orchestra.run.smithi031.stdout:2676 read 0x5133c60 thru 0x51365f2 (0x2993 bytes) 2024-09-13T09:08:19.655 INFO:teuthology.orchestra.run.smithi031.stdout:2677 read 0xc9ae6f9 thru 0xc9be5a3 (0xfeab bytes) 2024-09-13T09:08:19.655 INFO:teuthology.orchestra.run.smithi031.stdout:2679 write 0x9054a5d thru 0x905d1a7 (0x874b bytes) 2024-09-13T09:08:19.657 INFO:teuthology.orchestra.run.smithi031.stdout:2680 write 0xd52a72d thru 0xd533dc0 (0x9694 bytes) 2024-09-13T09:08:19.659 INFO:teuthology.orchestra.run.smithi031.stdout:2681 read 0x235f871 thru 0x2365923 (0x60b3 bytes) 2024-09-13T09:08:19.660 INFO:teuthology.orchestra.run.smithi031.stdout:2683 read 0x51766b1 thru 0x517adde (0x472e bytes) 2024-09-13T09:08:19.660 INFO:teuthology.orchestra.run.smithi031.stdout:2685 punch from 0xc6ae4fe to 0xc6b45fe, (0x6100 bytes) 2024-09-13T09:08:19.660 INFO:teuthology.orchestra.run.smithi031.stdout:2687 read 0x1439ca5 thru 0x144628e (0xc5ea bytes) 2024-09-13T09:08:19.661 INFO:teuthology.orchestra.run.smithi031.stdout:2688 read 0xad40960 thru 0xad44849 (0x3eea bytes) 2024-09-13T09:08:19.661 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.662 INFO:teuthology.orchestra.run.smithi031.stdout:2690 write 0x35c84c6 thru 0x35cac8c (0x27c7 bytes) 2024-09-13T09:08:19.664 INFO:teuthology.orchestra.run.smithi031.stdout:2691 write 0x42d4ebb thru 0x42e4217 (0xf35d bytes) 2024-09-13T09:08:19.666 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.666 INFO:teuthology.orchestra.run.smithi031.stdout:2693 read 0x2bd4f8 thru 0x2cbd11 (0xe81a bytes) 2024-09-13T09:08:19.666 INFO:teuthology.orchestra.run.smithi031.stdout:2695 write 0x1a2e957 thru 0x1a37338 (0x89e2 bytes) 2024-09-13T09:08:19.668 INFO:teuthology.orchestra.run.smithi031.stdout:2696 read 0x4bfa1e thru 0x4c9e75 (0xa458 bytes) 2024-09-13T09:08:19.669 INFO:teuthology.orchestra.run.smithi031.stdout:2699 read 0xac6a81e thru 0xac794ff (0xece2 bytes) 2024-09-13T09:08:19.670 INFO:teuthology.orchestra.run.smithi031.stdout:2702 write 0x171e479 thru 0x17240f8 (0x5c80 bytes) 2024-09-13T09:08:19.673 INFO:teuthology.orchestra.run.smithi031.stdout:2703 writesame 0x9f82db4 thru 0x9f8cae4 data_size 0x1(0x9d31 bytes) 2024-09-13T09:08:19.735 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:19.731+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:19.738 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:19.738 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:19.741 INFO:teuthology.orchestra.run.smithi031.stdout:2704 writesame 0x7d205b6 thru 0x7d27651 data_size 0x2(0x709c bytes) 2024-09-13T09:08:19.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:19.764 INFO:teuthology.orchestra.run.smithi031.stdout:2706 write 0x7632659 thru 0x7633908 (0x12b0 bytes) 2024-09-13T09:08:19.779 INFO:teuthology.orchestra.run.smithi031.stdout:2707 trunc from 0xd725ab0 to 0x9815f71 2024-09-13T09:08:19.830 INFO:teuthology.orchestra.run.smithi031.stdout:2708 writesame 0xceb00e4 thru 0xceb8f7b data_size 0x4e(0x8e98 bytes) 2024-09-13T09:08:19.833 INFO:teuthology.orchestra.run.smithi031.stdout:2709 write 0x7d965f6 thru 0x7d96ba7 (0x5b2 bytes) 2024-09-13T09:08:19.835 INFO:teuthology.orchestra.run.smithi031.stdout:2711 read 0x245ad05 thru 0x24605c8 (0x58c4 bytes) 2024-09-13T09:08:19.836 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.836 INFO:teuthology.orchestra.run.smithi031.stdout:2713 read 0xa5529b7 thru 0xa562130 (0xf77a bytes) 2024-09-13T09:08:19.836 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:19.831+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:19.837 INFO:teuthology.orchestra.run.smithi031.stdout:2714 read 0x46c3e83 thru 0x46ce9df (0xab5d bytes) 2024-09-13T09:08:19.838 INFO:teuthology.orchestra.run.smithi031.stdout:2716 write 0x57ae929 thru 0x57b0d8b (0x2463 bytes) 2024-09-13T09:08:19.841 INFO:teuthology.orchestra.run.smithi031.stdout:2717 writesame 0x99ef4dd thru 0x99f0bb8 data_size 0x1c(0x16dc bytes) 2024-09-13T09:08:19.844 INFO:teuthology.orchestra.run.smithi031.stdout:2718 read 0x7252600 thru 0x72532c5 (0xcc6 bytes) 2024-09-13T09:08:19.863 INFO:teuthology.orchestra.run.smithi031.stdout:2720 write 0xebe39cd thru 0xebef1d0 (0xb804 bytes) 2024-09-13T09:08:19.866 INFO:teuthology.orchestra.run.smithi031.stdout:2721 writesame 0x8f11916 thru 0x8f1a913 data_size 0x7(0x8ffe bytes) 2024-09-13T09:08:19.914 INFO:teuthology.orchestra.run.smithi031.stdout:2722 trunc from 0xebef1d1 to 0xa19389 2024-09-13T09:08:19.920 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:19.920 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:19.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:19.937 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:19.933+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:19.977 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.977 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:19.977 INFO:teuthology.orchestra.run.smithi031.stdout:2726 read 0x13693d thru 0x138117 (0x17db bytes) 2024-09-13T09:08:19.977 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.027 INFO:teuthology.orchestra.run.smithi031.stdout:2728 write 0xed54d2c thru 0xed5adbe (0x6093 bytes) 2024-09-13T09:08:20.033 INFO:teuthology.orchestra.run.smithi031.stdout:2729 writesame 0x4be8624 thru 0x4bea7c3 data_size 0x10(0x21a0 bytes) 2024-09-13T09:08:20.038 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:20.033+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:20.040 INFO:teuthology.orchestra.run.smithi031.stdout:2731 read 0xdc8affb thru 0xdc8c508 (0x150e bytes) 2024-09-13T09:08:20.041 INFO:teuthology.orchestra.run.smithi031.stdout:2732 writesame 0xeb0cb53 thru 0xeb12217 data_size 0x1(0x56c5 bytes) 2024-09-13T09:08:20.060 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:20.060 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:20.075 INFO:teuthology.orchestra.run.smithi031.stdout:2733 read 0x3252388 thru 0x325d6c1 (0xb33a bytes) 2024-09-13T09:08:20.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:20.077 INFO:teuthology.orchestra.run.smithi031.stdout:2737 writesame 0xd13c95d thru 0xd13ce1a data_size 0x1(0x4be bytes) 2024-09-13T09:08:20.087 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.087 INFO:teuthology.orchestra.run.smithi031.stdout:2739 write 0x791cb36 thru 0x79256f2 (0x8bbd bytes) 2024-09-13T09:08:20.099 INFO:teuthology.orchestra.run.smithi031.stdout:2740 trunc from 0xed5adbf to 0xa6c7aaf 2024-09-13T09:08:20.113 INFO:teuthology.orchestra.run.smithi031.stdout:2741 writesame 0x2a3b37e thru 0x2a44567 data_size 0x13(0x91ea bytes) 2024-09-13T09:08:20.119 INFO:teuthology.orchestra.run.smithi031.stdout:2742 punch from 0x3559e1a to 0x355f804, (0x59ea bytes) 2024-09-13T09:08:20.119 INFO:teuthology.orchestra.run.smithi031.stdout:2743 write 0x8a83b7 thru 0x8aa692 (0x22dc bytes) 2024-09-13T09:08:20.121 INFO:teuthology.orchestra.run.smithi031.stdout:2745 punch from 0x210e30b to 0x2116551, (0x8246 bytes) 2024-09-13T09:08:20.121 INFO:teuthology.orchestra.run.smithi031.stdout:2746 read 0x8ee7dcd thru 0x8eef90c (0x7b40 bytes) 2024-09-13T09:08:20.128 INFO:teuthology.orchestra.run.smithi031.stdout:2747 trunc from 0xa6c7aaf to 0xda63ae3 2024-09-13T09:08:20.136 INFO:teuthology.orchestra.run.smithi031.stdout:2748 read 0x1fa0156 thru 0x1fa89fb (0x88a6 bytes) 2024-09-13T09:08:20.137 INFO:teuthology.orchestra.run.smithi031.stdout:2750 read 0x72023e0 thru 0x7205e8b (0x3aac bytes) 2024-09-13T09:08:20.138 INFO:teuthology.orchestra.run.smithi031.stdout:2752 write 0xd918f1a thru 0xd923db0 (0xae97 bytes) 2024-09-13T09:08:20.138 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:20.135+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:20.141 INFO:teuthology.orchestra.run.smithi031.stdout:2754 punch from 0x72b9a7f to 0x72bf282, (0x5803 bytes) 2024-09-13T09:08:20.141 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.144 INFO:teuthology.orchestra.run.smithi031.stdout:2756 trunc from 0xda63ae3 to 0xbfe9bc7 2024-09-13T09:08:20.158 INFO:teuthology.orchestra.run.smithi031.stdout:2757 read 0xa8f7d8f thru 0xa907b1a (0xfd8c bytes) 2024-09-13T09:08:20.158 INFO:teuthology.orchestra.run.smithi031.stdout:2758 read 0x5a6c5e4 thru 0x5a6ff95 (0x39b2 bytes) 2024-09-13T09:08:20.159 INFO:teuthology.orchestra.run.smithi031.stdout:2759 writesame 0x4ed6a57 thru 0x4ee0ab0 data_size 0xa(0xa05a bytes) 2024-09-13T09:08:20.167 INFO:teuthology.orchestra.run.smithi031.stdout:2760 write 0xbee663a thru 0xbef2a67 (0xc42e bytes) 2024-09-13T09:08:20.180 INFO:teuthology.orchestra.run.smithi031.stdout:2761 writesame 0xd75d516 thru 0xd763601 data_size 0x2(0x60ec bytes) 2024-09-13T09:08:20.182 INFO:teuthology.orchestra.run.smithi031.stdout:2762 read 0x644d095 thru 0x645bedb (0xee47 bytes) 2024-09-13T09:08:20.183 INFO:teuthology.orchestra.run.smithi031.stdout:2763 read 0xa692f1e thru 0xa69bba0 (0x8c83 bytes) 2024-09-13T09:08:20.187 INFO:teuthology.orchestra.run.smithi031.stdout:2764 trunc from 0xd763602 to 0xbb98ff7 2024-09-13T09:08:20.200 INFO:teuthology.orchestra.run.smithi031.stdout:2766 writesame 0x939753d thru 0x939a81a data_size 0x11(0x32de bytes) 2024-09-13T09:08:20.215 INFO:teuthology.orchestra.run.smithi031.stdout:2767 trunc from 0xbb98ff7 to 0x5b6c7b7 2024-09-13T09:08:20.222 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:20.222 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:20.231 INFO:teuthology.orchestra.run.smithi031.stdout:2768 punch from 0x2d91053 to 0x2d9cb9f, (0xbb4c bytes) 2024-09-13T09:08:20.231 INFO:teuthology.orchestra.run.smithi031.stdout:2769 writesame 0x58bb0d1 thru 0x58c87b3 data_size 0xb(0xd6e3 bytes) 2024-09-13T09:08:20.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:20.240 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:20.233+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:20.266 INFO:teuthology.orchestra.run.smithi031.stdout:2770 write 0xe7cc9a4 thru 0xe7cf204 (0x2861 bytes) 2024-09-13T09:08:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.271 INFO:teuthology.orchestra.run.smithi031.stdout:2774 read 0x6b1aae1 thru 0x6b276a2 (0xcbc2 bytes) 2024-09-13T09:08:20.289 INFO:teuthology.orchestra.run.smithi031.stdout:2775 trunc from 0xe7cf205 to 0x5df163f 2024-09-13T09:08:20.302 INFO:teuthology.orchestra.run.smithi031.stdout:2776 write 0x297dec6 thru 0x29889d3 (0xab0e bytes) 2024-09-13T09:08:20.304 INFO:teuthology.orchestra.run.smithi031.stdout:2779 read 0x4b3235c thru 0x4b3b4d1 (0x9176 bytes) 2024-09-13T09:08:20.322 INFO:teuthology.orchestra.run.smithi031.stdout:2780 write 0xb6e9d5a thru 0xb6ed832 (0x3ad9 bytes) 2024-09-13T09:08:20.325 INFO:teuthology.orchestra.run.smithi031.stdout:2782 read 0x7f3776b thru 0x7f445bd (0xce53 bytes) 2024-09-13T09:08:20.333 INFO:teuthology.orchestra.run.smithi031.stdout:2783 write 0xbd6ac6a thru 0xbd6f93b (0x4cd2 bytes) 2024-09-13T09:08:20.339 INFO:teuthology.orchestra.run.smithi031.stdout:2784 trunc from 0xbd6f93c to 0x8dd6c04 2024-09-13T09:08:20.340 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:20.333+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:20.352 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.364 INFO:teuthology.orchestra.run.smithi031.stdout:2786 writesame 0xad29817 thru 0xad2e3a3 data_size 0x15(0x4b8d bytes) 2024-09-13T09:08:20.365 INFO:teuthology.orchestra.run.smithi031.stdout:2787 punch from 0xbf47da to 0xbfcd46, (0x856c bytes) 2024-09-13T09:08:20.365 INFO:teuthology.orchestra.run.smithi031.stdout:2788 write 0xa3e2988 thru 0xa3e3085 (0x6fe bytes) 2024-09-13T09:08:20.366 INFO:teuthology.orchestra.run.smithi031.stdout:2789 writesame 0xad17a19 thru 0xad1c207 data_size 0x5(0x47ef bytes) 2024-09-13T09:08:20.386 INFO:teuthology.orchestra.run.smithi031.stdout:2790 write 0xca68f39 thru 0xca724dc (0x95a4 bytes) 2024-09-13T09:08:20.389 INFO:teuthology.orchestra.run.smithi031.stdout:2791 read 0x690e1dc thru 0x6916c4b (0x8a70 bytes) 2024-09-13T09:08:20.390 INFO:teuthology.orchestra.run.smithi031.stdout:2793 punch from 0x696b961 to 0x696cdd4, (0x1473 bytes) 2024-09-13T09:08:20.390 INFO:teuthology.orchestra.run.smithi031.stdout:2795 write 0xac6f5fb thru 0xac7df1b (0xe921 bytes) 2024-09-13T09:08:20.392 INFO:teuthology.orchestra.run.smithi031.stdout:2798 punch from 0x4a273e5 to 0x4a2dbdc, (0x67f7 bytes) 2024-09-13T09:08:20.392 INFO:teuthology.orchestra.run.smithi031.stdout:2799 punch from 0x1396585 to 0x139cc97, (0x6712 bytes) 2024-09-13T09:08:20.409 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:20.409 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:20.411 INFO:teuthology.orchestra.run.smithi031.stdout:2800 trunc from 0xca724dd to 0x2d68d49 2024-09-13T09:08:20.425 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:20.441 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:20.437+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:20.453 INFO:teuthology.orchestra.run.smithi031.stdout:2802 write 0xecaa55a thru 0xecabd7b (0x1822 bytes) 2024-09-13T09:08:20.454 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:20.455 INFO:teuthology.orchestra.run.smithi031.stdout:2804 read 0x6259b34 thru 0x625c5f2 (0x2abf bytes) 2024-09-13T09:08:20.455 INFO:teuthology.orchestra.run.smithi031.stdout:2805 clone 25 order 23 su 262144 sc 8 2024-09-13T09:08:20.541 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:20.537+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:20.578 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:20.578 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:20.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:20.679 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:20.675+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:20.743 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:20.737+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:20.763 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:20.763 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:20.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:20.843 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:20.841+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:20.944 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:20.941+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:20.951 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:20.951 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:20.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:21.045 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:21.041+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:21.059 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:08:21.060 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:08:21.060 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:08:21.060 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-13T09:08:21.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T09:08:21.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:08:21.104 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:21.104 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:21.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:21.146 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:21.141+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:21.185 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone24 2024-09-13T09:08:21.247 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:21.241+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:21.296 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:21.296 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:21.314 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:21.348 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:21.341+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:21.385 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-13T09:08:21.449 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:21.445+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:21.482 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:21.483 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:21.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:21.538 INFO:teuthology.orchestra.run.smithi031.stdout:2806 writesame 0x4789c41 thru 0x478d938 data_size 0x4(0x3cf8 bytes) 2024-09-13T09:08:21.550 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:21.545+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:21.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:21.560 INFO:teuthology.orchestra.run.smithi031.stdout:2807 punch from 0x346effb to 0x347069a, (0x169f bytes) 2024-09-13T09:08:21.576 INFO:teuthology.orchestra.run.smithi031.stdout:2808 trunc from 0xecabd7c to 0x3a25431 2024-09-13T09:08:21.640 INFO:teuthology.orchestra.run.smithi031.stdout:2809 trunc from 0x3a25431 to 0x736ae16 2024-09-13T09:08:21.651 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:21.647+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:21.651 INFO:teuthology.orchestra.run.smithi031.stdout:2810 write 0x41c5b0 thru 0x4272ab (0xacfc bytes) 2024-09-13T09:08:21.671 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:21.671 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:21.673 INFO:teuthology.orchestra.run.smithi031.stdout:2812 write 0xd2da263 thru 0xd2dbc2f (0x19cd bytes) 2024-09-13T09:08:21.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:21.704 INFO:teuthology.orchestra.run.smithi031.stdout:2813 trunc from 0xd2dbc30 to 0xb69613 2024-09-13T09:08:21.726 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:21.726 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:21.731 INFO:teuthology.orchestra.run.smithi031.stdout:2815 read 0xb559c2 thru 0xb64178 (0xe7b7 bytes) 2024-09-13T09:08:21.752 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:21.745+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:21.753 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:21.753 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:21.761 INFO:teuthology.orchestra.run.smithi031.stdout:2816 write 0xb9a180f thru 0xb9acb6a (0xb35c bytes) 2024-09-13T09:08:21.770 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:21.770 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:21.779 INFO:teuthology.orchestra.run.smithi031.stdout:2817 write 0xbed59a9 thru 0xbed7e33 (0x248b bytes) 2024-09-13T09:08:21.786 INFO:teuthology.orchestra.run.smithi031.stdout:2818 write 0x68caa7a thru 0x68cd78f (0x2d16 bytes) 2024-09-13T09:08:21.788 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:21.789 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:21.792 INFO:teuthology.orchestra.run.smithi031.stdout:2820 read 0x4d144a9 thru 0x4d2040b (0xbf63 bytes) 2024-09-13T09:08:21.793 INFO:teuthology.orchestra.run.smithi031.stdout:2822 read 0x1848a8c thru 0x1855c7b (0xd1f0 bytes) 2024-09-13T09:08:21.793 INFO:teuthology.orchestra.run.smithi031.stdout:2823 read 0x9abb144 thru 0x9acafe0 (0xfe9d bytes) 2024-09-13T09:08:21.809 INFO:teuthology.orchestra.run.smithi031.stdout:2824 trunc from 0xbed7e34 to 0x15d23ea 2024-09-13T09:08:21.810 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:21.810 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:21.834 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:21.834 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:21.834 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:21.835 INFO:teuthology.orchestra.run.smithi031.stdout:2825 punch from 0xca697a to 0xca7474, (0xafa bytes) 2024-09-13T09:08:21.835 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:21.835 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:21.849 INFO:teuthology.orchestra.run.smithi031.stdout:2827 trunc from 0x15d23ea to 0xa343d1f 2024-09-13T09:08:21.853 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:21.849+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:21.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:21.861 INFO:teuthology.orchestra.run.smithi031.stdout:2829 writesame 0x4866e9 thru 0x48c2cb data_size 0x1(0x5be3 bytes) 2024-09-13T09:08:21.863 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:21.863 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:21.877 INFO:teuthology.orchestra.run.smithi031.stdout:2832 writesame 0x5e265aa thru 0x5e2b3d7 data_size 0x1(0x4e2e bytes) 2024-09-13T09:08:21.885 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:21.885 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:21.893 INFO:teuthology.orchestra.run.smithi031.stdout:2833 read 0xdfcbf0 thru 0xe00380 (0x3791 bytes) 2024-09-13T09:08:21.895 INFO:teuthology.orchestra.run.smithi031.stdout:2834 punch from 0x9497536 to 0x94a5d2c, (0xe7f6 bytes) 2024-09-13T09:08:21.895 INFO:teuthology.orchestra.run.smithi031.stdout:2835 punch from 0x6cd55b6 to 0x6ce4b77, (0xf5c1 bytes) 2024-09-13T09:08:21.895 INFO:teuthology.orchestra.run.smithi031.stdout:2837 write 0x5c5ba1a thru 0x5c69fb8 (0xe59f bytes) 2024-09-13T09:08:21.914 INFO:teuthology.orchestra.run.smithi031.stdout:2838 trunc from 0xa343d1f to 0x297ec97 2024-09-13T09:08:21.943 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:21.954 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:21.949+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:2841 write 0x69210f0 thru 0x692f9d4 (0xe8e5 bytes) 2024-09-13T09:08:21.969 INFO:teuthology.orchestra.run.smithi031.stdout:2842 trunc from 0x692f9d5 to 0x5432349 2024-09-13T09:08:21.984 INFO:teuthology.orchestra.run.smithi031.stdout:2844 writesame 0xd290f8 thru 0xd3577e data_size 0x3(0xc687 bytes) 2024-09-13T09:08:21.990 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:21.990 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:21.991 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:21.999 INFO:teuthology.orchestra.run.smithi031.stdout:2845 trunc from 0x5432349 to 0x486ff8f 2024-09-13T09:08:22.008 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:22.018 INFO:teuthology.orchestra.run.smithi031.stdout:2846 read 0x4168a9e thru 0x4168d1e (0x281 bytes) 2024-09-13T09:08:22.018 INFO:teuthology.orchestra.run.smithi031.stdout:2847 punch from 0xf860bb to 0xf8e8a9, (0x87ee bytes) 2024-09-13T09:08:22.018 INFO:teuthology.orchestra.run.smithi031.stdout:2849 punch from 0x3cd5e37 to 0x3ce55e1, (0xf7aa bytes) 2024-09-13T09:08:22.018 INFO:teuthology.orchestra.run.smithi031.stdout:2851 read 0xc7e9db thru 0xc8867c (0x9ca2 bytes) 2024-09-13T09:08:22.018 INFO:teuthology.orchestra.run.smithi031.stdout:2852 read 0x437312d thru 0x4377d89 (0x4c5d bytes) 2024-09-13T09:08:22.019 INFO:teuthology.orchestra.run.smithi031.stdout:2853 read 0x2a33e24 thru 0x2a3a7f9 (0x69d6 bytes) 2024-09-13T09:08:22.019 INFO:teuthology.orchestra.run.smithi031.stdout:2854 writesame 0x4255b58 thru 0x4255c61 data_size 0x7(0x10a bytes) 2024-09-13T09:08:22.025 INFO:teuthology.orchestra.run.smithi031.stdout:2855 read 0x253c757 thru 0x253e41b (0x1cc5 bytes) 2024-09-13T09:08:22.052 INFO:teuthology.orchestra.run.smithi031.stdout:2859 write 0xc7ae100 thru 0xc7bc6e5 (0xe5e6 bytes) 2024-09-13T09:08:22.054 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:22.047+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:22.059 INFO:teuthology.orchestra.run.smithi031.stdout:2860 write 0x4966f41 thru 0x496733e (0x3fe bytes) 2024-09-13T09:08:22.065 INFO:teuthology.orchestra.run.smithi031.stdout:2861 write 0x3faf31b thru 0x3fb0b1e (0x1804 bytes) 2024-09-13T09:08:22.068 INFO:teuthology.orchestra.run.smithi031.stdout:2862 read 0x19ca4fe thru 0x19ca608 (0x10b bytes) 2024-09-13T09:08:22.069 INFO:teuthology.orchestra.run.smithi031.stdout:2863 write 0x1209b3d thru 0x12114e9 (0x79ad bytes) 2024-09-13T09:08:22.071 INFO:teuthology.orchestra.run.smithi031.stdout:2865 read 0x7202448 thru 0x7209f49 (0x7b02 bytes) 2024-09-13T09:08:22.085 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:08:22.086 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T09:08:22.091 INFO:teuthology.orchestra.run.smithi031.stdout:2866 writesame 0xca679ca thru 0xca6b90e data_size 0x1(0x3f45 bytes) 2024-09-13T09:08:22.106 INFO:teuthology.orchestra.run.smithi031.stdout:2867 write 0xcce82f1 thru 0xccefe2f (0x7b3f bytes) 2024-09-13T09:08:22.113 INFO:teuthology.orchestra.run.smithi031.stdout:2868 read 0xfbfd36 thru 0xfce690 (0xe95b bytes) 2024-09-13T09:08:22.128 INFO:teuthology.orchestra.run.smithi031.stdout:2869 write 0xd9a3d21 thru 0xd9b3ab1 (0xfd91 bytes) 2024-09-13T09:08:22.133 INFO:teuthology.orchestra.run.smithi031.stdout:2870 writesame 0xb09b066 thru 0xb0a34a8 data_size 0x7(0x8443 bytes) 2024-09-13T09:08:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:2872 read 0x91d2260 thru 0x91d8b0e (0x68af bytes) 2024-09-13T09:08:22.142 INFO:teuthology.orchestra.run.smithi031.stdout:2873 write 0x4f00cd0 thru 0x4f045a5 (0x38d6 bytes) 2024-09-13T09:08:22.146 INFO:teuthology.orchestra.run.smithi031.stdout:2874 read 0x911ede4 thru 0x91244b3 (0x56d0 bytes) 2024-09-13T09:08:22.155 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:22.151+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:22.158 INFO:teuthology.orchestra.run.smithi031.stdout:2875 trunc from 0xd9b3ab2 to 0x61df118 2024-09-13T09:08:22.160 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:22.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.028652116999993, 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:08:22.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:22.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:08:22.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.200369937000001, 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:08:22.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56973 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4501504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.911464841000001, 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091397946999999993, 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56973 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247918+0000", 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247921+0000", 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247927+0000", 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2389999999999999e-06 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247932+0000", 2024-09-13T09:08:22.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339077+0000", 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091144366000000004 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339203+0000", 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126919 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339317+0000", 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113205 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56975 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4515840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.911271073999998, 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091417921999999999, 2024-09-13T09:08:22.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56975 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247936+0000", 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999824 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248117+0000", 2024-09-13T09:08:22.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180827 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248127+0000", 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248133+0000", 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.19e-06 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339347+0000", 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091213581000000002 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339419+0000", 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.203e-05 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339530+0000", 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111359 2024-09-13T09:08:22.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56976 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3681280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.911227676999999, 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091677039000000002, 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56976 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248155+0000", 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248159+0000", 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248166+0000", 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999999e-06 2024-09-13T09:08:22.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248172+0000", 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339558+0000", 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091385818999999993 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339705+0000", 2024-09-13T09:08:22.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014693599999999999 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339833+0000", 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012808200000000001 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56990 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3795968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.908087174999999, 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091546566999999995, 2024-09-13T09:08:22.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56990 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251298+0000", 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.041e-06 2024-09-13T09:08:22.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251303+0000", 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251319+0000", 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5852e-05 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342591+0000", 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091271605000000006 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342728+0000", 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013662799999999999 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342843+0000", 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115442 2024-09-13T09:08:22.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57022 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4334592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.908004548000001, 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091691988000000002, 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57022 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251382+0000", 2024-09-13T09:08:22.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5659999999999998e-06 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251391+0000", 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0170000000000002e-06 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251397+0000", 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.908e-06 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342870+0000", 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091473410000000005 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342945+0000", 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4175e-05 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343071+0000", 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012638499999999999 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57026 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5113856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.907981611, 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091880383999999996, 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57026 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251404+0000", 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251410+0000", 2024-09-13T09:08:22.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251415+0000", 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2129999999999999e-06 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343098+0000", 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091683073000000004 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343171+0000", 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3149e-05 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343282+0000", 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110899 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57027 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4348928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.907963968000001, 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092079082000000007, 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57027 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251419+0000", 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251421+0000", 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251427+0000", 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4419999999999997e-06 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251432+0000", 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7620000000000003e-06 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343310+0000", 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091878572000000006 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343383+0000", 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2368999999999998e-05 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343499+0000", 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116142 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57028 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5128192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.907946623000001, 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092372671000000003, 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57028 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251439+0000", 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1730000000000002e-06 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251445+0000", 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251450+0000", 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343526+0000", 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092076621999999997 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343605+0000", 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8830000000000002e-05 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343810+0000", 2024-09-13T09:08:22.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204544 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.907930099000001, 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:22.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:08:22.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.907850301, 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:08:22.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.907601230000001, 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:08:22.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:08:22.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.907288089000001, 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:08:22.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:08:22.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.906834857, 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:22.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:08:22.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.906810343, 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.906545114, 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:08:22.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:08:22.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.906257265000001, 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57037 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.905821064000001, 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092241045999999993, 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57037 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253152+0000", 2024-09-13T09:08:22.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995894 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253567+0000", 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000414886 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253576+0000", 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6750000000000008e-06 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253582+0000", 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345610+0000", 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092027943000000001 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345682+0000", 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2311999999999995e-05 2024-09-13T09:08:22.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345804+0000", 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121332 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.905795960999999, 2024-09-13T09:08:22.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:22.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:08:22.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:08:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:22.257 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:22.257 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:22.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:22.259 INFO:teuthology.orchestra.run.smithi031.stdout:2876 write 0xc006f2f thru 0xc007015 (0xe7 bytes) 2024-09-13T09:08:22.260 INFO:teuthology.orchestra.run.smithi031.stdout:2877 write 0x237c1fe thru 0x237dc87 (0x1a8a bytes) 2024-09-13T09:08:22.260 INFO:teuthology.orchestra.run.smithi031.stdout:2878 write 0x4207cc4 thru 0x4214111 (0xc44e bytes) 2024-09-13T09:08:22.260 INFO:teuthology.orchestra.run.smithi031.stdout:2879 read 0x94ace2a thru 0x94b92ba (0xc491 bytes) 2024-09-13T09:08:22.260 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.260 INFO:teuthology.orchestra.run.smithi031.stdout:2881 writesame 0x970e5b6 thru 0x971620b data_size 0x2(0x7c56 bytes) 2024-09-13T09:08:22.260 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:22.251+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:22.265 INFO:teuthology.orchestra.run.smithi031.stdout:2882 write 0xe3be071 thru 0xe3be402 (0x392 bytes) 2024-09-13T09:08:22.271 INFO:teuthology.orchestra.run.smithi031.stdout:2883 read 0x65eb447 thru 0x65f43d6 (0x8f90 bytes) 2024-09-13T09:08:22.271 INFO:teuthology.orchestra.run.smithi031.stdout:2885 writesame 0x6944e23 thru 0x6953dfa data_size 0x64(0xefd8 bytes) 2024-09-13T09:08:22.275 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.275 INFO:teuthology.orchestra.run.smithi031.stdout:2888 read 0xa2ecdd1 thru 0xa2f1c3a (0x4e6a bytes) 2024-09-13T09:08:22.275 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.275 INFO:teuthology.orchestra.run.smithi031.stdout:2892 read 0x6b0236f thru 0x6b058d2 (0x3564 bytes) 2024-09-13T09:08:22.276 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.276 INFO:teuthology.orchestra.run.smithi031.stdout:2895 read 0x645cfdb thru 0x6461e11 (0x4e37 bytes) 2024-09-13T09:08:22.276 INFO:teuthology.orchestra.run.smithi031.stdout:2897 write 0x9a3912 thru 0x9ae2eb (0xa9da bytes) 2024-09-13T09:08:22.279 INFO:teuthology.orchestra.run.smithi031.stdout:2898 read 0x1e14d07 thru 0x1e18ec8 (0x41c2 bytes) 2024-09-13T09:08:22.279 INFO:teuthology.orchestra.run.smithi031.stdout:2899 read 0x616ff0d thru 0x617ddab (0xde9f bytes) 2024-09-13T09:08:22.280 INFO:teuthology.orchestra.run.smithi031.stdout:2900 write 0x6f6b89b thru 0x6f788c4 (0xd02a bytes) 2024-09-13T09:08:22.283 INFO:teuthology.orchestra.run.smithi031.stdout:2901 write 0xc174b61 thru 0xc178cb4 (0x4154 bytes) 2024-09-13T09:08:22.287 INFO:teuthology.orchestra.run.smithi031.stdout:2902 writesame 0x9f570a3 thru 0x9f57aa5 data_size 0xb(0xa03 bytes) 2024-09-13T09:08:22.293 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.293 INFO:teuthology.orchestra.run.smithi031.stdout:2904 read 0x7f8b719 thru 0x7f9b320 (0xfc08 bytes) 2024-09-13T09:08:22.293 INFO:teuthology.orchestra.run.smithi031.stdout:2907 writesame 0x537f092 thru 0x538c447 data_size 0x9(0xd3b6 bytes) 2024-09-13T09:08:22.305 INFO:teuthology.orchestra.run.smithi031.stdout:2908 read 0x20faa96 thru 0x2103160 (0x86cb bytes) 2024-09-13T09:08:22.306 INFO:teuthology.orchestra.run.smithi031.stdout:2909 writesame 0x69e4887 thru 0x69e5d47 data_size 0x21(0x14c1 bytes) 2024-09-13T09:08:22.312 INFO:teuthology.orchestra.run.smithi031.stdout:2910 read 0x523b942 thru 0x523cb48 (0x1207 bytes) 2024-09-13T09:08:22.313 INFO:teuthology.orchestra.run.smithi031.stdout:2912 write 0x4341fa4 thru 0x4348075 (0x60d2 bytes) 2024-09-13T09:08:22.315 INFO:teuthology.orchestra.run.smithi031.stdout:2914 writesame 0xb5cd376 thru 0xb5ced26 data_size 0x1(0x19b1 bytes) 2024-09-13T09:08:22.327 INFO:teuthology.orchestra.run.smithi031.stdout:2915 read 0x8681fda thru 0x8683ec4 (0x1eeb bytes) 2024-09-13T09:08:22.328 INFO:teuthology.orchestra.run.smithi031.stdout:2917 read 0xc188156 thru 0xc19558c (0xd437 bytes) 2024-09-13T09:08:22.329 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.329 INFO:teuthology.orchestra.run.smithi031.stdout:2919 read 0x3e8e805 thru 0x3e92dd8 (0x45d4 bytes) 2024-09-13T09:08:22.330 INFO:teuthology.orchestra.run.smithi031.stdout:2921 write 0x68f74cf thru 0x68f8d86 (0x18b8 bytes) 2024-09-13T09:08:22.336 INFO:teuthology.orchestra.run.smithi031.stdout:2922 writesame 0xb3ebe0 thru 0xb3f0af data_size 0x7(0x4d0 bytes) 2024-09-13T09:08:22.340 INFO:teuthology.orchestra.run.smithi031.stdout:2923 write 0x8caaf55 thru 0x8cb9e9e (0xef4a bytes) 2024-09-13T09:08:22.345 INFO:teuthology.orchestra.run.smithi031.stdout:2928 punch from 0x4d122e3 to 0x4d163aa, (0x40c7 bytes) 2024-09-13T09:08:22.355 INFO:teuthology.orchestra.run.smithi031.stdout:2929 trunc from 0xe3be403 to 0xb067a09 2024-09-13T09:08:22.359 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:22.355+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:22.397 INFO:teuthology.orchestra.run.smithi031.stdout:2931 trunc from 0xb067a09 to 0x61455d5 2024-09-13T09:08:22.409 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:22.409 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:22.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:104924 3.1b 3:d9a0ca8e:::rbd_data.1124cc1b8ca9.000000000000000c:head [truncate 4456448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:22.408609+0000", 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00017436700000000001, 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020015199999999999, 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 104924 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.410 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:22.408609+0000", 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:22.408609+0000", 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:22.408607+0000", 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:22.408621+0000", 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4167e-05 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:22.408658+0000", 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7163000000000003e-05 2024-09-13T09:08:22.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:22.408681+0000", 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2263e-05 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:22.408747+0000", 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6086000000000004e-05 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:22.408820+0000", 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3061999999999999e-05 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 1 2024-09-13T09:08:22.414 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:22.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:22.449 INFO:teuthology.orchestra.run.smithi031.stdout:2932 write 0xb2e180d thru 0xb2f1629 (0xfe1d bytes) 2024-09-13T09:08:22.455 INFO:teuthology.orchestra.run.smithi031.stdout:2933 read 0x6836ee1 thru 0x683c167 (0x5287 bytes) 2024-09-13T09:08:22.460 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:22.455+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:22.485 INFO:teuthology.orchestra.run.smithi031.stdout:2934 trunc from 0xb2f162a to 0x4c567d 2024-09-13T09:08:22.526 INFO:teuthology.orchestra.run.smithi031.stdout:2937 trunc from 0x4c567d to 0x6f2ed2b 2024-09-13T09:08:22.537 INFO:teuthology.orchestra.run.smithi031.stdout:2939 read 0x31874c0 thru 0x31874c9 (0xa bytes) 2024-09-13T09:08:22.538 INFO:teuthology.orchestra.run.smithi031.stdout:2940 punch from 0x4913980 to 0x4921a37, (0xe0b7 bytes) 2024-09-13T09:08:22.549 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:22.549 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:22.550 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:22.550 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:22.550 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:22.560 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:22.555+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:22.563 INFO:teuthology.orchestra.run.smithi031.stdout:2941 write 0xce46801 thru 0xce4a98d (0x418d bytes) 2024-09-13T09:08:22.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:22.571 INFO:teuthology.orchestra.run.smithi031.stdout:2942 read 0x820c63a thru 0x821b6b4 (0xf07b bytes) 2024-09-13T09:08:22.571 INFO:teuthology.orchestra.run.smithi031.stdout:2944 writesame 0x1aa2987 thru 0x1ab0b43 data_size 0x3(0xe1bd bytes) 2024-09-13T09:08:22.605 INFO:teuthology.orchestra.run.smithi031.stdout:2947 write 0xb88a4b2 thru 0xb88cf74 (0x2ac3 bytes) 2024-09-13T09:08:22.612 INFO:teuthology.orchestra.run.smithi031.stdout:2948 write 0xb3c38fb thru 0xb3c47fb (0xf01 bytes) 2024-09-13T09:08:22.617 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.617 INFO:teuthology.orchestra.run.smithi031.stdout:2950 punch from 0x6b6daf2 to 0x6b76114, (0x8622 bytes) 2024-09-13T09:08:22.618 INFO:teuthology.orchestra.run.smithi031.stdout:2951 read 0xbd2a5f0 thru 0xbd39c94 (0xf6a5 bytes) 2024-09-13T09:08:22.618 INFO:teuthology.orchestra.run.smithi031.stdout:2952 read 0x40882dc thru 0x409226d (0x9f92 bytes) 2024-09-13T09:08:22.618 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.618 INFO:teuthology.orchestra.run.smithi031.stdout:2955 write 0xc109e9e thru 0xc11427b (0xa3de bytes) 2024-09-13T09:08:22.634 INFO:teuthology.orchestra.run.smithi031.stdout:2956 write 0xda10e13 thru 0xda167a5 (0x5993 bytes) 2024-09-13T09:08:22.639 INFO:teuthology.orchestra.run.smithi031.stdout:2957 writesame 0x8abd0c4 thru 0x8ac3fd4 data_size 0x1(0x6f11 bytes) 2024-09-13T09:08:22.661 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:22.655+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:22.695 INFO:teuthology.orchestra.run.smithi031.stdout:2958 write 0x3f2b5be thru 0x3f2fddc (0x481f bytes) 2024-09-13T09:08:22.699 INFO:teuthology.orchestra.run.smithi031.stdout:2959 read 0xad2ad39 thru 0xad2fec0 (0x5188 bytes) 2024-09-13T09:08:22.699 INFO:teuthology.orchestra.run.smithi031.stdout:2960 writesame 0x1929401 thru 0x1931b8c data_size 0x32(0x878c bytes) 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673978834, 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:08:22.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673931501, 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673873965, 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:08:22.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.713 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673822479, 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.714 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:08:22.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.718 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673775537000001, 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673728144, 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:08:22.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673680869, 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:08:22.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673634669, 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:08:22.732 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:08:22.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.736 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.673570664, 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.737 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:08:22.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.741 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.671099575, 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:22.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.671043853, 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:08:22.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:08:22.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.67099395, 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:08:22.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:08:22.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:08:22.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.670946894, 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:08:22.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.759 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.670898411, 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.760 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:22.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.670833146, 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:08:22.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:08:22.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.670786230000001, 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:08:22.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:08:22.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.670698782000001, 2024-09-13T09:08:22.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:08:22.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.669236933000001, 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:08:22.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.669158847, 2024-09-13T09:08:22.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:08:22.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.786 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.669106758, 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:22.787 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:08:22.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:22.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:22.791 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:22.793 INFO:teuthology.orchestra.run.smithi031.stdout:2961 writesame 0xa2d660c thru 0xa2dc4cc data_size 0x1(0x5ec1 bytes) 2024-09-13T09:08:22.793 INFO:teuthology.orchestra.run.smithi031.stdout:2962 writesame 0xbd42364 thru 0xbd4a77f data_size 0x26(0x841c bytes) 2024-09-13T09:08:22.793 INFO:teuthology.orchestra.run.smithi031.stdout:2963 trunc from 0xda167a6 to 0xcaee4ca 2024-09-13T09:08:22.793 INFO:teuthology.orchestra.run.smithi031.stdout:2966 write 0xbf9bdfc thru 0xbfa3bdc (0x7de1 bytes) 2024-09-13T09:08:22.794 INFO:teuthology.orchestra.run.smithi031.stdout:2967 write 0xc566856 thru 0xc57085d (0xa008 bytes) 2024-09-13T09:08:22.794 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:22.787+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:22.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:22.799 INFO:teuthology.orchestra.run.smithi031.stdout:2968 writesame 0x8a9ff89 thru 0x8aa021c data_size 0x14(0x294 bytes) 2024-09-13T09:08:22.803 INFO:teuthology.orchestra.run.smithi031.stdout:2969 write 0x1e6eb3f thru 0x1e7087d (0x1d3f bytes) 2024-09-13T09:08:22.806 INFO:teuthology.orchestra.run.smithi031.stdout:2970 read 0x7725c66 thru 0x77281e3 (0x257e bytes) 2024-09-13T09:08:22.806 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.806 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:22.837 INFO:teuthology.orchestra.run.smithi031.stdout:2974 trunc from 0xcaee4ca to 0x14f9f9f 2024-09-13T09:08:22.862 INFO:teuthology.orchestra.run.smithi031.stdout:2975 read 0x1302141 thru 0x1305d50 (0x3c10 bytes) 2024-09-13T09:08:22.863 INFO:teuthology.orchestra.run.smithi031.stdout:2976 read 0x70104f thru 0x7040c7 (0x3079 bytes) 2024-09-13T09:08:22.894 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:22.887+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:22.904 INFO:teuthology.orchestra.run.smithi031.stdout:2977 write 0xb1c0dd2 thru 0xb1c83e7 (0x7616 bytes) 2024-09-13T09:08:22.911 INFO:teuthology.orchestra.run.smithi031.stdout:2978 write 0x76f97a5 thru 0x76fbb62 (0x23be bytes) 2024-09-13T09:08:22.932 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:22.933 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:22.942 INFO:teuthology.orchestra.run.smithi031.stdout:2980 trunc from 0xb1c83e8 to 0x266f1dc 2024-09-13T09:08:22.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:22.967 INFO:teuthology.orchestra.run.smithi031.stdout:2981 read 0x2915af thru 0x29964d (0x809f bytes) 2024-09-13T09:08:22.984 INFO:teuthology.orchestra.run.smithi031.stdout:2982 write 0x510acc3 thru 0x5115615 (0xa953 bytes) 2024-09-13T09:08:22.990 INFO:teuthology.orchestra.run.smithi031.stdout:2983 punch from 0x2c9bfd4 to 0x2ca7038, (0xb064 bytes) 2024-09-13T09:08:22.990 INFO:teuthology.orchestra.run.smithi031.stdout:2984 read 0x1138815 thru 0x11479a5 (0xf191 bytes) 2024-09-13T09:08:22.991 INFO:teuthology.orchestra.run.smithi031.stdout:2986 punch from 0x31cbacd to 0x31d6873, (0xada6 bytes) 2024-09-13T09:08:22.991 INFO:teuthology.orchestra.run.smithi031.stdout:2987 read 0x7651c4 thru 0x7731e2 (0xe01f bytes) 2024-09-13T09:08:22.992 INFO:teuthology.orchestra.run.smithi031.stdout:2988 read 0xb3ec55 thru 0xb44f4a (0x62f6 bytes) 2024-09-13T09:08:22.994 INFO:teuthology.orchestra.run.smithi031.stdout:2989 punch from 0x1a90e73 to 0x1a99c38, (0x8dc5 bytes) 2024-09-13T09:08:22.994 INFO:teuthology.orchestra.run.smithi031.stdout:2990 write 0x13a5ede thru 0x13a883f (0x2962 bytes) 2024-09-13T09:08:22.994 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:22.989+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:23.015 INFO:teuthology.orchestra.run.smithi031.stdout:2991 write 0x8e66fdc thru 0x8e6f286 (0x82ab bytes) 2024-09-13T09:08:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:2992 punch from 0x87b4c67 to 0x87c303b, (0xe3d4 bytes) 2024-09-13T09:08:23.019 INFO:teuthology.orchestra.run.smithi031.stdout:2993 write 0x2892703 thru 0x289ce4f (0xa74d bytes) 2024-09-13T09:08:23.041 INFO:teuthology.orchestra.run.smithi031.stdout:2994 writesame 0xbc6ae7a thru 0xbc70c78 data_size 0xd(0x5dff bytes) 2024-09-13T09:08:23.042 INFO:teuthology.orchestra.run.smithi031.stdout:2995 read 0x9a2359d thru 0x9a2fbe3 (0xc647 bytes) 2024-09-13T09:08:23.043 INFO:teuthology.orchestra.run.smithi031.stdout:2997 punch from 0x4b2fd5 to 0x4be63f, (0xb66a bytes) 2024-09-13T09:08:23.043 INFO:teuthology.orchestra.run.smithi031.stdout:2998 writesame 0x1ee86ff thru 0x1ef217d data_size 0x1(0x9a7f bytes) 2024-09-13T09:08:23.082 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:23.083 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:23.083 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:23.083 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:23.083 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:23.096 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:23.091+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:23.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:23.113 INFO:teuthology.orchestra.run.smithi031.stdout:2999 read 0x8696629 thru 0x869ec62 (0x863a bytes) 2024-09-13T09:08:23.113 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.122 INFO:teuthology.orchestra.run.smithi031.stdout:3003 trunc from 0xbc70c79 to 0x88bd75b 2024-09-13T09:08:23.151 INFO:teuthology.orchestra.run.smithi031.stdout:3004 write 0x92ced9a thru 0x92dbd3d (0xcfa4 bytes) 2024-09-13T09:08:23.157 INFO:teuthology.orchestra.run.smithi031.stdout:3005 write 0x778642b thru 0x778c9b3 (0x6589 bytes) 2024-09-13T09:08:23.178 INFO:teuthology.orchestra.run.smithi031.stdout:3006 trunc from 0x92dbd3e to 0xeb3b30f 2024-09-13T09:08:23.189 INFO:teuthology.orchestra.run.smithi031.stdout:3007 read 0x457b055 thru 0x458641e (0xb3ca bytes) 2024-09-13T09:08:23.190 INFO:teuthology.orchestra.run.smithi031.stdout:3009 writesame 0x2c81dda thru 0x2c87a12 data_size 0x1(0x5c39 bytes) 2024-09-13T09:08:23.196 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:23.193+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:23.231 INFO:teuthology.orchestra.run.smithi031.stdout:3011 write 0xb0662d thru 0xb0f4cf (0x8ea3 bytes) 2024-09-13T09:08:23.234 INFO:teuthology.orchestra.run.smithi031.stdout:3014 write 0x66c1ccf thru 0x66c4abf (0x2df1 bytes) 2024-09-13T09:08:23.243 INFO:teuthology.orchestra.run.smithi031.stdout:3015 punch from 0x1d4e8ab to 0x1d5c1bb, (0xd910 bytes) 2024-09-13T09:08:23.243 INFO:teuthology.orchestra.run.smithi031.stdout:3016 read 0xda8d90a thru 0xda9cdf4 (0xf4eb bytes) 2024-09-13T09:08:23.243 INFO:teuthology.orchestra.run.smithi031.stdout:3018 write 0xa2e4ca7 thru 0xa2ef07b (0xa3d5 bytes) 2024-09-13T09:08:23.248 INFO:teuthology.orchestra.run.smithi031.stdout:3019 write 0xe9a78fa thru 0xe9b1391 (0x9a98 bytes) 2024-09-13T09:08:23.255 INFO:teuthology.orchestra.run.smithi031.stdout:3020 writesame 0xe8a9dcf thru 0xe8b133f data_size 0x7(0x7571 bytes) 2024-09-13T09:08:23.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.269 INFO:teuthology.orchestra.run.smithi031.stdout:3030 write 0x1bc02b1 thru 0x1bc3922 (0x3672 bytes) 2024-09-13T09:08:23.272 INFO:teuthology.orchestra.run.smithi031.stdout:3032 writesame 0x466bbf8 thru 0x467279d data_size 0x9(0x6ba6 bytes) 2024-09-13T09:08:23.277 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:23.277 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:23.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.657183883000002, 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.278 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:08:23.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.657160552999997, 2024-09-13T09:08:23.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.657137536, 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.290 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.657114405999998, 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.291 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:08:23.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.654497642999999, 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.654473557000003, 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.299 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.654446903999997, 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:08:23.303 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:08:23.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.654423264000002, 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:23.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.650607723, 2024-09-13T09:08:23.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:23.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:08:23.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.650584180000003, 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.317 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:23.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.321 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.650560890000001, 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:08:23.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.650537376999999, 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.326 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:23.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.330 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.650478921000001, 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:08:23.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:08:23.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:23.335 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.646447985000002, 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:23.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.326549688, 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.340 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:08:23.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:23.344 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.324177908999999, 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:08:23.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.323643994000001, 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:08:23.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:23.354 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.320243471000001, 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.319243663999998, 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.359 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:23.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:08:23.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.316073385999999, 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:08:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:08:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:08:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:08:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.369 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:23.370 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.370 INFO:teuthology.orchestra.run.smithi031.stdout:3034 trunc from 0xeb3b30f to 0x6b2ea8a 2024-09-13T09:08:23.370 INFO:teuthology.orchestra.run.smithi031.stdout:3035 write 0x8f3125c thru 0x8f3406a (0x2e0f bytes) 2024-09-13T09:08:23.371 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:23.363+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:23.371 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:23.375 INFO:teuthology.orchestra.run.smithi031.stdout:3036 write 0xb3bfa28 thru 0xb3c976f (0x9d48 bytes) 2024-09-13T09:08:23.385 INFO:teuthology.orchestra.run.smithi031.stdout:3037 writesame 0x1d8bfda thru 0x1d98ee9 data_size 0x8(0xcf10 bytes) 2024-09-13T09:08:23.398 INFO:teuthology.orchestra.run.smithi031.stdout:3038 writesame 0xa37a400 thru 0xa388fbd data_size 0x55(0xebbe bytes) 2024-09-13T09:08:23.408 INFO:teuthology.orchestra.run.smithi031.stdout:3041 punch from 0x4391cbb to 0x439fdff, (0xe144 bytes) 2024-09-13T09:08:23.408 INFO:teuthology.orchestra.run.smithi031.stdout:3042 read 0x6531614 thru 0x653aeda (0x98c7 bytes) 2024-09-13T09:08:23.409 INFO:teuthology.orchestra.run.smithi031.stdout:3043 read 0x31f608a thru 0x320322e (0xd1a5 bytes) 2024-09-13T09:08:23.410 INFO:teuthology.orchestra.run.smithi031.stdout:3044 read 0x7d4a860 thru 0x7d53f0e (0x96af bytes) 2024-09-13T09:08:23.410 INFO:teuthology.orchestra.run.smithi031.stdout:3045 writesame 0x24f37bf thru 0x24f6c51 data_size 0x1(0x3493 bytes) 2024-09-13T09:08:23.434 INFO:teuthology.orchestra.run.smithi031.stdout:3046 read 0x1a8e483 thru 0x1a9c7ec (0xe36a bytes) 2024-09-13T09:08:23.436 INFO:teuthology.orchestra.run.smithi031.stdout:3048 read 0x563ea6 thru 0x5724a5 (0xe600 bytes) 2024-09-13T09:08:23.437 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.454 INFO:teuthology.orchestra.run.smithi031.stdout:3051 trunc from 0xb3c9770 to 0x4fa5294 2024-09-13T09:08:23.471 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:23.465+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:23.478 INFO:teuthology.orchestra.run.smithi031.stdout:3052 read 0x159d5ff thru 0x15a42b1 (0x6cb3 bytes) 2024-09-13T09:08:23.479 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.479 INFO:teuthology.orchestra.run.smithi031.stdout:3056 read 0x2658702 thru 0x2662cd0 (0xa5cf bytes) 2024-09-13T09:08:23.480 INFO:teuthology.orchestra.run.smithi031.stdout:3057 read 0x533f97 thru 0x5343be (0x428 bytes) 2024-09-13T09:08:23.481 INFO:teuthology.orchestra.run.smithi031.stdout:3058 write 0x3adcb52 thru 0x3ae2722 (0x5bd1 bytes) 2024-09-13T09:08:23.484 INFO:teuthology.orchestra.run.smithi031.stdout:3061 read 0xd05742 thru 0xd0b687 (0x5f46 bytes) 2024-09-13T09:08:23.486 INFO:teuthology.orchestra.run.smithi031.stdout:3064 trunc from 0x4fa5294 to 0x4963d2a 2024-09-13T09:08:23.500 INFO:teuthology.orchestra.run.smithi031.stdout:3065 read 0x2f9fef5 thru 0x2faadcf (0xaedb bytes) 2024-09-13T09:08:23.501 INFO:teuthology.orchestra.run.smithi031.stdout:3066 read 0x1456429 thru 0x14582ee (0x1ec6 bytes) 2024-09-13T09:08:23.501 INFO:teuthology.orchestra.run.smithi031.stdout:3067 punch from 0xccd936 to 0xcd110d, (0x37d7 bytes) 2024-09-13T09:08:23.501 INFO:teuthology.orchestra.run.smithi031.stdout:3068 read 0x13b0fcc thru 0x13b1dc1 (0xdf6 bytes) 2024-09-13T09:08:23.515 INFO:teuthology.orchestra.run.smithi031.stdout:3071 write 0x56aba24 thru 0x56ad0f1 (0x16ce bytes) 2024-09-13T09:08:23.521 INFO:teuthology.orchestra.run.smithi031.stdout:3073 read 0x1af894 thru 0x1b581e (0x5f8b bytes) 2024-09-13T09:08:23.523 INFO:teuthology.orchestra.run.smithi031.stdout:3075 writesame 0x4ee240b thru 0x4ee9d88 data_size 0x1(0x797e bytes) 2024-09-13T09:08:23.557 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:23.557 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:23.557 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:23.557 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:23.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:23.572 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:23.567+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:23.576 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.592 INFO:teuthology.orchestra.run.smithi031.stdout:3078 trunc from 0x56ad0f2 to 0xb5a7ff7 2024-09-13T09:08:23.603 INFO:teuthology.orchestra.run.smithi031.stdout:3079 read 0x811424c thru 0x8121152 (0xcf07 bytes) 2024-09-13T09:08:23.603 INFO:teuthology.orchestra.run.smithi031.stdout:3080 write 0x22520e6 thru 0x2254729 (0x2644 bytes) 2024-09-13T09:08:23.606 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.606 INFO:teuthology.orchestra.run.smithi031.stdout:3084 write 0x83fcac9 thru 0x8409d7a (0xd2b2 bytes) 2024-09-13T09:08:23.613 INFO:teuthology.orchestra.run.smithi031.stdout:3085 read 0x51f1687 thru 0x51f54c6 (0x3e40 bytes) 2024-09-13T09:08:23.613 INFO:teuthology.orchestra.run.smithi031.stdout:3087 read 0x22aa8d7 thru 0x22b89be (0xe0e8 bytes) 2024-09-13T09:08:23.614 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.616 INFO:teuthology.orchestra.run.smithi031.stdout:3089 trunc from 0xb5a7ff7 to 0xa9dc491 2024-09-13T09:08:23.630 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.630 INFO:teuthology.orchestra.run.smithi031.stdout:3093 punch from 0xa1023a1 to 0xa10a309, (0x7f68 bytes) 2024-09-13T09:08:23.630 INFO:teuthology.orchestra.run.smithi031.stdout:3094 punch from 0xdff4de to 0xe0ddd5, (0xe8f7 bytes) 2024-09-13T09:08:23.630 INFO:teuthology.orchestra.run.smithi031.stdout:3095 read 0x650ae86 thru 0x651a8d6 (0xfa51 bytes) 2024-09-13T09:08:23.630 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.636 INFO:teuthology.orchestra.run.smithi031.stdout:3099 trunc from 0xa9dc491 to 0xca03256 2024-09-13T09:08:23.644 INFO:teuthology.orchestra.run.smithi031.stdout:3100 punch from 0x153ca0c to 0x1546c4a, (0xa23e bytes) 2024-09-13T09:08:23.644 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.663 INFO:teuthology.orchestra.run.smithi031.stdout:3102 trunc from 0xca03256 to 0x5c191f1 2024-09-13T09:08:23.673 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:23.669+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:23.714 INFO:teuthology.orchestra.run.smithi031.stdout:3103 write 0xb970e14 thru 0xb976aac (0x5c99 bytes) 2024-09-13T09:08:23.720 INFO:teuthology.orchestra.run.smithi031.stdout:3104 write 0x5d605bd thru 0x5d660a8 (0x5aec bytes) 2024-09-13T09:08:23.726 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:23.726 INFO:teuthology.orchestra.run.smithi031.stdout:3107 write 0x34e1435 thru 0x34ea338 (0x8f04 bytes) 2024-09-13T09:08:23.729 INFO:teuthology.orchestra.run.smithi031.stdout:3108 punch from 0x3fe3858 to 0x3fe709d, (0x3845 bytes) 2024-09-13T09:08:23.729 INFO:teuthology.orchestra.run.smithi031.stdout:3110 read 0xad871cf thru 0xad8f82d (0x865f bytes) 2024-09-13T09:08:23.729 INFO:teuthology.orchestra.run.smithi031.stdout:3111 write 0xb39adf2 thru 0xb3a8ae7 (0xdcf6 bytes) 2024-09-13T09:08:23.735 INFO:teuthology.orchestra.run.smithi031.stdout:3113 write 0x1981ecc thru 0x1990dcd (0xef02 bytes) 2024-09-13T09:08:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:3115 write 0x72477f8 thru 0x724c590 (0x4d99 bytes) 2024-09-13T09:08:23.742 INFO:teuthology.orchestra.run.smithi031.stdout:3116 write 0x22e2caa thru 0x22f1713 (0xea6a bytes) 2024-09-13T09:08:23.746 INFO:teuthology.orchestra.run.smithi031.stdout:3118 read 0x4a19580 thru 0x4a1c3a3 (0x2e24 bytes) 2024-09-13T09:08:23.746 INFO:teuthology.orchestra.run.smithi031.stdout:3119 punch from 0x42c1ec to 0x43901d, (0xce31 bytes) 2024-09-13T09:08:23.746 INFO:teuthology.orchestra.run.smithi031.stdout:3121 write 0x9cc29df thru 0x9cc50fc (0x271e bytes) 2024-09-13T09:08:23.749 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:23.750 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:23.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:23.781 INFO:teuthology.orchestra.run.smithi031.stdout:3122 trunc from 0xb976aad to 0xfdece1 2024-09-13T09:08:23.806 INFO:teuthology.orchestra.run.smithi031.stdout:3123 read 0x732c83 thru 0x7411f7 (0xe575 bytes) 2024-09-13T09:08:23.807 INFO:teuthology.orchestra.run.smithi031.stdout:3124 read 0xbf6663 thru 0xbf7428 (0xdc6 bytes) 2024-09-13T09:08:23.808 INFO:teuthology.orchestra.run.smithi031.stdout:3125 read 0x438966 thru 0x4413da (0x8a75 bytes) 2024-09-13T09:08:23.809 INFO:teuthology.orchestra.run.smithi031.stdout:3127 clone 26 order 23 su 524288 sc 10 2024-09-13T09:08:23.809 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:23.805+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:23.874 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:23.867+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:23.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487684956000003, 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487660372999997, 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487636369000001, 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487612886000001, 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487589479, 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487566229999999, 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487534124, 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487498506999998, 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487466527000002, 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487397248000001, 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487372188000002, 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487348758000003, 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487325374999998, 2024-09-13T09:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487301959, 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487278035999999, 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487244951999998, 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487221151999996, 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487196826000002, 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487164266000001, 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:08:24.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.487124393000002, 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.023 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:24.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:08:24.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:24.027 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:24.029 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:24.029 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:24.023+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:24.068 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:24.068 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:24.084 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:24.084 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:24.102 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:24.102 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:24.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:08:24.120 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:24.120 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:24.130 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:24.125+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:24.144 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:24.145 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:24.165 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:24.166 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:24.167 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:24.167 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:24.167 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:24.167 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:24.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:24.187 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:24.187 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:24.196 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone25 intact 2024-09-13T09:08:24.208 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:24.208 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:24.249 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-13T09:08:24.268 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:24.265+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:24.331 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:24.325+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:24.350 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:24.350 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:24.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:24.431 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:24.427+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:24.532 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:24.529+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:24.536 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:24.536 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:24.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:24.634 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:24.629+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:24.688 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:24.688 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:24.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:24.735 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:24.729+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:24.753 INFO:teuthology.orchestra.run.smithi031.stdout:3128 write 0x2a13732 thru 0x2a1ebdd (0xb4ac bytes) 2024-09-13T09:08:24.781 INFO:teuthology.orchestra.run.smithi031.stdout:3129 write 0xc0e5a13 thru 0xc0f3477 (0xda65 bytes) 2024-09-13T09:08:24.795 INFO:teuthology.orchestra.run.smithi031.stdout:3130 trunc from 0xc0f3478 to 0x6b9709b 2024-09-13T09:08:24.811 INFO:teuthology.orchestra.run.smithi031.stdout:3132 write 0x2380d0c thru 0x2389eb2 (0x91a7 bytes) 2024-09-13T09:08:24.821 INFO:teuthology.orchestra.run.smithi031.stdout:3134 trunc from 0x6b9709b to 0x9020949 2024-09-13T09:08:24.828 INFO:teuthology.orchestra.run.smithi031.stdout:3135 write 0x28af580 thru 0x28b095a (0x13db bytes) 2024-09-13T09:08:24.836 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:24.831+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:24.837 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:24.837 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:24.846 INFO:teuthology.orchestra.run.smithi031.stdout:3136 writesame 0xe4bc4c7 thru 0xe4c35c3 data_size 0x41(0x70fd bytes) 2024-09-13T09:08:24.846 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:24.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:24.867 INFO:teuthology.orchestra.run.smithi031.stdout:3138 trunc from 0xe4c35c4 to 0x1c9167e 2024-09-13T09:08:24.896 INFO:teuthology.orchestra.run.smithi031.stdout:3139 read 0x16624da thru 0x1666d7c (0x48a3 bytes) 2024-09-13T09:08:24.897 INFO:teuthology.orchestra.run.smithi031.stdout:3140 punch from 0x1467de3 to 0x14681d7, (0x3f4 bytes) 2024-09-13T09:08:24.897 INFO:teuthology.orchestra.run.smithi031.stdout:3141 read 0x16e3e8d thru 0x16f330c (0xf480 bytes) 2024-09-13T09:08:24.912 INFO:teuthology.orchestra.run.smithi031.stdout:3143 write 0x5ebc447 thru 0x5ec5ae2 (0x969c bytes) 2024-09-13T09:08:24.917 INFO:teuthology.orchestra.run.smithi031.stdout:3146 read 0x48832bb thru 0x488d088 (0x9dce bytes) 2024-09-13T09:08:24.918 INFO:teuthology.orchestra.run.smithi031.stdout:3147 read 0x97fb32 thru 0x98c7cc (0xcc9b bytes) 2024-09-13T09:08:24.919 INFO:teuthology.orchestra.run.smithi031.stdout:3148 read 0x4b83502 thru 0x4b8a5da (0x70d9 bytes) 2024-09-13T09:08:24.919 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:24.919 INFO:teuthology.orchestra.run.smithi031.stdout:3151 write 0x5c9562 thru 0x5d4f47 (0xb9e6 bytes) 2024-09-13T09:08:24.921 INFO:teuthology.orchestra.run.smithi031.stdout:3152 read 0x161138e thru 0x1619b8e (0x8801 bytes) 2024-09-13T09:08:24.937 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:24.931+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:24.941 INFO:teuthology.orchestra.run.smithi031.stdout:3153 write 0xcc16055 thru 0xcc1eb76 (0x8b22 bytes) 2024-09-13T09:08:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:3154 punch from 0x2fbfdd2 to 0x2fcfa99, (0xfcc7 bytes) 2024-09-13T09:08:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:3155 punch from 0xc7b3a13 to 0xc7be9fb, (0xafe8 bytes) 2024-09-13T09:08:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:24.948 INFO:teuthology.orchestra.run.smithi031.stdout:3158 write 0x220ebe5 thru 0x221be90 (0xd2ac bytes) 2024-09-13T09:08:24.950 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:24.950 INFO:teuthology.orchestra.run.smithi031.stdout:3160 writesame 0x94e3d05 thru 0x94eeba5 data_size 0x1(0xaea1 bytes) 2024-09-13T09:08:24.995 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:24.995 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:25.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:25.012 INFO:teuthology.orchestra.run.smithi031.stdout:3161 writesame 0x77b2e3 thru 0x77c949 data_size 0x5(0x1667 bytes) 2024-09-13T09:08:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:3162 read 0x6482f92 thru 0x648fe9b (0xcf0a bytes) 2024-09-13T09:08:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:3163 read 0x71c733b thru 0x71d2de5 (0xbaab bytes) 2024-09-13T09:08:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:3164 trunc from 0xcc1eb77 to 0xa0eddb2 2024-09-13T09:08:25.038 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:25.033+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:25.042 INFO:teuthology.orchestra.run.smithi031.stdout:3165 read 0x4140c89 thru 0x414c84f (0xbbc7 bytes) 2024-09-13T09:08:25.043 INFO:teuthology.orchestra.run.smithi031.stdout:3166 punch from 0x57c1c7b to 0x57ce358, (0xc6dd bytes) 2024-09-13T09:08:25.043 INFO:teuthology.orchestra.run.smithi031.stdout:3167 write 0x53fb81 thru 0x53fbac (0x2c bytes) 2024-09-13T09:08:25.046 INFO:teuthology.orchestra.run.smithi031.stdout:3168 punch from 0x83a1c0b to 0x83aeb7d, (0xcf72 bytes) 2024-09-13T09:08:25.049 INFO:teuthology.orchestra.run.smithi031.stdout:3171 trunc from 0xa0eddb2 to 0x8b39eca 2024-09-13T09:08:25.059 INFO:teuthology.orchestra.run.smithi031.stdout:3172 read 0x1dbe4f0 thru 0x1dccf8d (0xea9e bytes) 2024-09-13T09:08:25.060 INFO:teuthology.orchestra.run.smithi031.stdout:3173 write 0x4fbd52c thru 0x4fcc63f (0xf114 bytes) 2024-09-13T09:08:25.064 INFO:teuthology.orchestra.run.smithi031.stdout:3174 punch from 0x405ba6f to 0x405dd3b, (0x22cc bytes) 2024-09-13T09:08:25.073 INFO:teuthology.orchestra.run.smithi031.stdout:3176 write 0x951fdeb thru 0x952d533 (0xd749 bytes) 2024-09-13T09:08:25.079 INFO:teuthology.orchestra.run.smithi031.stdout:3177 read 0x4e51762 thru 0x4e5d1a0 (0xba3f bytes) 2024-09-13T09:08:25.089 INFO:teuthology.orchestra.run.smithi031.stdout:3178 write 0x996bf4e thru 0x9979ee3 (0xdf96 bytes) 2024-09-13T09:08:25.095 INFO:teuthology.orchestra.run.smithi031.stdout:3179 write 0xe86ba5 thru 0xe86e75 (0x2d1 bytes) 2024-09-13T09:08:25.098 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:3182 writesame 0x2cc4b77 thru 0x2cd34be data_size 0x14(0xe948 bytes) 2024-09-13T09:08:25.106 INFO:teuthology.orchestra.run.smithi031.stdout:3183 read 0x319f97a thru 0x31ab525 (0xbbac bytes) 2024-09-13T09:08:25.107 INFO:teuthology.orchestra.run.smithi031.stdout:3184 read 0x593d02f thru 0x594702b (0x9ffd bytes) 2024-09-13T09:08:25.109 INFO:teuthology.orchestra.run.smithi031.stdout:3186 writesame 0x46c4699 thru 0x46cdbb8 data_size 0x10(0x9520 bytes) 2024-09-13T09:08:25.114 INFO:teuthology.orchestra.run.smithi031.stdout:3187 punch from 0x96c2f58 to 0x96cd4ed, (0xa595 bytes) 2024-09-13T09:08:25.119 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:08:25.128 INFO:teuthology.orchestra.run.smithi031.stdout:3188 write 0xa9ef326 thru 0xa9f1442 (0x211d bytes) 2024-09-13T09:08:25.139 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:25.135+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:25.147 INFO:teuthology.orchestra.run.smithi031.stdout:3189 write 0xd3f5cb2 thru 0xd3fc86e (0x6bbd bytes) 2024-09-13T09:08:25.153 INFO:teuthology.orchestra.run.smithi031.stdout:3190 punch from 0x43b646c to 0x43bfbf1, (0x9785 bytes) 2024-09-13T09:08:25.153 INFO:teuthology.orchestra.run.smithi031.stdout:3192 clone 27 order 24 su 65536 sc 2 2024-09-13T09:08:25.166 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:25.166 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:25.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:25.240 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:25.237+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:25.341 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:25.337+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:25.343 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:25.343 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:25.358 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:25.443 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:25.435+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:25.479 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:25.479 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:25.494 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:25.579 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:25.575+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:25.643 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:25.637+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:25.656 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:25.656 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:25.672 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:25.744 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:25.741+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:25.843 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:25.844 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:25.845 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:25.841+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:25.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:25.946 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:25.939+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:25.982 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:25.982 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:25.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:26.083 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:26.083+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:26.147 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:26.147+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:26.178 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:26.178 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:26.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:26.228 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:26.248 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:26.241+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:26.252 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone26 from 0xd3fc86f (overlap 0xfdece1) to 0x19647c 2024-09-13T09:08:26.349 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:26.347+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:26.366 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:26.366 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:26.380 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-13T09:08:26.383 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:26.402 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:26.403 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:26.414 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:26.414 INFO:teuthology.orchestra.run.smithi031.stdout:3195 write 0x1b8a75b thru 0x1b97c60 (0xd506 bytes) 2024-09-13T09:08:26.420 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:26.421 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:26.449 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:26.447+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:26.451 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:26.451 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:26.469 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:26.469 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:26.479 INFO:teuthology.orchestra.run.smithi031.stdout:3200 trunc from 0xd3fc86f to 0xd98db25 2024-09-13T09:08:26.490 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:26.490 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:26.490 INFO:teuthology.orchestra.run.smithi031.stdout:3201 write 0xb8b38d0 thru 0xb8b6ed0 (0x3601 bytes) 2024-09-13T09:08:26.511 INFO:teuthology.orchestra.run.smithi031.stdout:3202 trunc from 0xd98db25 to 0x7fef0fb 2024-09-13T09:08:26.512 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:26.512 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:26.540 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:26.540 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:26.547 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:26.547 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.414670131999998, 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:26.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:08:26.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:08:26.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:08:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:08:26.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:08:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:08:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:08:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:08:26.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:08:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:08:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:08:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.586387952000003, 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:08:26.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56973 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4501504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.297482855999998, 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091397946999999993, 2024-09-13T09:08:26.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56973 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247919+0000", 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247918+0000", 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247921+0000", 2024-09-13T09:08:26.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6249999999999999e-06 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247927+0000", 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2389999999999999e-06 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247932+0000", 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0849999999999996e-06 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339077+0000", 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091144366000000004 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339203+0000", 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126919 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339317+0000", 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113205 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56975 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4515840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.297289089, 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091417921999999999, 2024-09-13T09:08:26.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56975 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248112+0000", 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.247936+0000", 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999824 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248117+0000", 2024-09-13T09:08:26.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000180827 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248127+0000", 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9969999999999993e-06 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248133+0000", 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.19e-06 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339347+0000", 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091213581000000002 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339419+0000", 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.203e-05 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339530+0000", 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111359 2024-09-13T09:08:26.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56976 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3681280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.297245692000001, 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091677039000000002, 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56976 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248156+0000", 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248155+0000", 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248159+0000", 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2370000000000001e-06 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248166+0000", 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5809999999999999e-06 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.248172+0000", 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9580000000000004e-06 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339558+0000", 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091385818999999993 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339705+0000", 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014693599999999999 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.339833+0000", 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012808200000000001 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:56990 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 3795968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.29410519, 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091546566999999995, 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 56990 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251296+0000", 2024-09-13T09:08:26.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251298+0000", 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.041e-06 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251303+0000", 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3190000000000002e-06 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251319+0000", 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5852e-05 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342591+0000", 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091271605000000006 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342728+0000", 2024-09-13T09:08:26.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013662799999999999 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342843+0000", 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115442 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57022 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4334592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:26.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.294022562999999, 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091691988000000002, 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57022 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251379+0000", 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251382+0000", 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5659999999999998e-06 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251391+0000", 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0170000000000002e-06 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251397+0000", 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.908e-06 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342870+0000", 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091473410000000005 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.342945+0000", 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4175e-05 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343071+0000", 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012638499999999999 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57026 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5113856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.293999626000002, 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091880383999999996, 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57026 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251402+0000", 2024-09-13T09:08:26.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251404+0000", 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251410+0000", 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.207e-06 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251415+0000", 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2129999999999999e-06 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343098+0000", 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091683073000000004 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343171+0000", 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3149e-05 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343282+0000", 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110899 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57027 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4348928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.293981982999998, 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092079082000000007, 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57027 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251420+0000", 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251419+0000", 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251421+0000", 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0990000000000001e-06 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251427+0000", 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4419999999999997e-06 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251432+0000", 2024-09-13T09:08:26.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7620000000000003e-06 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343310+0000", 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.091878572000000006 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343383+0000", 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2368999999999998e-05 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343499+0000", 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116142 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57028 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5128192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.293964637999999, 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092372671000000003, 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57028 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251439+0000", 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1730000000000002e-06 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251445+0000", 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:26.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251450+0000", 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343526+0000", 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092076621999999997 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343605+0000", 2024-09-13T09:08:26.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8830000000000002e-05 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343810+0000", 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204544 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.293948113999999, 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:08:26.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:26.605 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.293868316000001, 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:08:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.610 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.293619244999999, 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:26.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:08:26.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:08:26.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.293306103999999, 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:08:26.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:08:26.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.292852872000001, 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:08:26.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.624 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.292828358000001, 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:08:26.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:08:26.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:08:26.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.292563129000001, 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:08:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:08:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:08:26.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:08:26.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.292275279999998, 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:08:26.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57037 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5185536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.291839078999999, 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092241045999999993, 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57037 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253563+0000", 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253152+0000", 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995894 2024-09-13T09:08:26.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253567+0000", 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000414886 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253576+0000", 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6750000000000008e-06 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253582+0000", 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2600000000000002e-06 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345610+0000", 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092027943000000001 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345682+0000", 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2311999999999995e-05 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345804+0000", 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000121332 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.291813976, 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:26.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:08:26.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:08:26.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:08:26.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:26.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:26.651 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:26.652 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:26.652 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:26.653 INFO:teuthology.orchestra.run.smithi031.stdout:3203 trunc from 0x7fef0fb to 0x58e55f0 2024-09-13T09:08:26.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:26.655 INFO:teuthology.orchestra.run.smithi031.stdout:3204 read 0x15329e6 thru 0x1537d09 (0x5324 bytes) 2024-09-13T09:08:26.655 INFO:teuthology.orchestra.run.smithi031.stdout:3205 writesame 0x4b8f2d9 thru 0x4b9e4f7 data_size 0x9(0xf21f bytes) 2024-09-13T09:08:26.655 INFO:teuthology.orchestra.run.smithi031.stdout:3206 write 0x3561b94 thru 0x3569bd4 (0x8041 bytes) 2024-09-13T09:08:26.655 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:26.645+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:26.681 INFO:teuthology.orchestra.run.smithi031.stdout:3207 writesame 0xedacad6 thru 0xedbbffa data_size 0x39(0xf525 bytes) 2024-09-13T09:08:26.681 INFO:teuthology.orchestra.run.smithi031.stdout:3208 read 0x9068332 thru 0x906e73c (0x640b bytes) 2024-09-13T09:08:26.681 INFO:teuthology.orchestra.run.smithi031.stdout:3209 read 0x216d0bc thru 0x217b3e1 (0xe326 bytes) 2024-09-13T09:08:26.682 INFO:teuthology.orchestra.run.smithi031.stdout:3210 write 0xc8286df thru 0xc82e302 (0x5c24 bytes) 2024-09-13T09:08:26.688 INFO:teuthology.orchestra.run.smithi031.stdout:3211 read 0x7cea4f6 thru 0x7ceec7b (0x4786 bytes) 2024-09-13T09:08:26.688 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:26.688 INFO:teuthology.orchestra.run.smithi031.stdout:3213 read 0x631fcde thru 0x632c370 (0xc693 bytes) 2024-09-13T09:08:26.688 INFO:teuthology.orchestra.run.smithi031.stdout:3217 read 0x23dc7cb thru 0x23e1511 (0x4d47 bytes) 2024-09-13T09:08:26.689 INFO:teuthology.orchestra.run.smithi031.stdout:3218 writesame 0x37770a8 thru 0x37853db data_size 0x25(0xe334 bytes) 2024-09-13T09:08:26.725 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:26.725 INFO:teuthology.orchestra.run.smithi031.stdout:3220 writesame 0xd9c8c60 thru 0xd9d0064 data_size 0x1(0x7405 bytes) 2024-09-13T09:08:26.753 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:26.751+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:26.769 INFO:teuthology.orchestra.run.smithi031.stdout:3222 punch from 0xd8d147d to 0xd8d33ff, (0x1f82 bytes) 2024-09-13T09:08:26.769 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:26.770 INFO:teuthology.orchestra.run.smithi031.stdout:3224 writesame 0x8a7af72 thru 0x8a82152 data_size 0x1(0x71e1 bytes) 2024-09-13T09:08:26.793 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:26.794 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:26.794 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:26.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:26.820 INFO:teuthology.orchestra.run.smithi031.stdout:3225 write 0xcdfcef4 thru 0xce07a56 (0xab63 bytes) 2024-09-13T09:08:26.846 INFO:teuthology.orchestra.run.smithi031.stdout:3226 trunc from 0xedbbffb to 0x6dd5785 2024-09-13T09:08:26.857 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:26.853+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:26.883 INFO:teuthology.orchestra.run.smithi031.stdout:3227 trunc from 0x6dd5785 to 0x8f8232e 2024-09-13T09:08:26.899 INFO:teuthology.orchestra.run.smithi031.stdout:3228 trunc from 0x8f8232e to 0x9f5a90b 2024-09-13T09:08:26.929 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:26.929 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:26.929 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:26.929 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:26.929 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:26.940 INFO:teuthology.orchestra.run.smithi031.stdout:3229 write 0xdc13f64 thru 0xdc17f9b (0x4038 bytes) 2024-09-13T09:08:26.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:26.947 INFO:teuthology.orchestra.run.smithi031.stdout:3230 read 0xdebce6 thru 0xdf87df (0xcafa bytes) 2024-09-13T09:08:26.949 INFO:teuthology.orchestra.run.smithi031.stdout:3231 read 0x660876a thru 0x660c3ed (0x3c84 bytes) 2024-09-13T09:08:26.949 INFO:teuthology.orchestra.run.smithi031.stdout:3232 write 0xc39665c thru 0xc399ffc (0x39a1 bytes) 2024-09-13T09:08:26.955 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:26.951+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:26.982 INFO:teuthology.orchestra.run.smithi031.stdout:3233 trunc from 0xdc17f9c to 0x2b66dc5 2024-09-13T09:08:27.008 INFO:teuthology.orchestra.run.smithi031.stdout:3234 read 0x159952f thru 0x159d379 (0x3e4b bytes) 2024-09-13T09:08:27.044 INFO:teuthology.orchestra.run.smithi031.stdout:3235 write 0xbbb1159 thru 0xbbba700 (0x95a8 bytes) 2024-09-13T09:08:27.056 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:27.055+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:27.059 INFO:teuthology.orchestra.run.smithi031.stdout:3236 trunc from 0xbbba701 to 0x8df63e9 2024-09-13T09:08:27.081 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.082 INFO:teuthology.orchestra.run.smithi031.stdout:3238 read 0x8aa23a9 thru 0x8aac9ad (0xa605 bytes) 2024-09-13T09:08:27.082 INFO:teuthology.orchestra.run.smithi031.stdout:3239 read 0x41e3ed2 thru 0x41e8ba6 (0x4cd5 bytes) 2024-09-13T09:08:27.082 INFO:teuthology.orchestra.run.smithi031.stdout:3241 read 0x798776b thru 0x7996c97 (0xf52d bytes) 2024-09-13T09:08:27.082 INFO:teuthology.orchestra.run.smithi031.stdout:3242 punch from 0x37c661b to 0x37ca9a7, (0x438c bytes) 2024-09-13T09:08:27.082 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.082 INFO:teuthology.orchestra.run.smithi031.stdout:3244 read 0x4466444 thru 0x446fa8e (0x964b bytes) 2024-09-13T09:08:27.087 INFO:teuthology.orchestra.run.smithi031.stdout:3245 trunc from 0x8df63e9 to 0x71b3bf3 2024-09-13T09:08:27.102 INFO:teuthology.orchestra.run.smithi031.stdout:3246 writesame 0x28ff8e4 thru 0x290b8d3 data_size 0x53(0xbff0 bytes) 2024-09-13T09:08:27.106 INFO:teuthology.orchestra.run.smithi031.stdout:3247 read 0x68f004 thru 0x693045 (0x4042 bytes) 2024-09-13T09:08:27.119 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:08:27.119 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:08:27.120 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T09:08:27.120 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.999485 2024-09-13T09:08:27.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.9994849959154812 2024-09-13T09:08:27.121 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:27.121 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:27.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:27.121 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:27.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.121 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092652715, 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:08:27.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092605381999999, 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.127 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:08:27.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:08:27.131 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092547845999999, 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.132 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.137 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092496359999998, 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:08:27.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.142 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092449418000001, 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.144 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:27.145 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.146 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:08:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092402024999998, 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:08:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:08:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:08:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:08:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:08:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:08:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:08:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092354749999998, 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:08:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:08:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092308549999998, 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:08:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:08:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:08:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:08:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.092244545, 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:08:27.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:08:27.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:08:27.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:08:27.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:08:27.173 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.089773456, 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:08:27.174 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:08:27.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:08:27.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:08:27.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.089717734000001, 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:08:27.180 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.181 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.181 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.181 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.181 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.181 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:08:27.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:08:27.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:08:27.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:08:27.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:08:27.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.089667831, 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.186 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:27.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:08:27.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.191 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.192 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.192 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.192 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:27.192 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.089620775, 2024-09-13T09:08:27.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:27.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:08:27.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:08:27.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:08:27.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:08:27.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:08:27.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:08:27.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:08:27.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.089572292, 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.199 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.200 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.200 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.200 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.200 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:08:27.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.200 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:27.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:08:27.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:08:27.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:08:27.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.205 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.205 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.089507027, 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:08:27.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.211 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.089460111000001, 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.212 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.213 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.213 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:08:27.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.213 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:08:27.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:08:27.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:08:27.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:08:27.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:08:27.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:08:27.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:08:27.219 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.219 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.219 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.089372662999999, 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.220 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:08:27.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:08:27.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:08:27.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:08:27.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.087910814000001, 2024-09-13T09:08:27.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:27.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:08:27.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:08:27.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.087832727999999, 2024-09-13T09:08:27.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:27.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:27.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:08:27.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:08:27.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:08:27.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:08:27.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 19.087780639000002, 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:08:27.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:08:27.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:08:27.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.241 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:27.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:08:27.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:27.245 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:27.237+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:3249 trunc from 0x71b3bf3 to 0xd1a706e 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:3250 writesame 0x5d600d4 thru 0x5d697d3 data_size 0x80(0x9700 bytes) 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:3251 trunc from 0xd1a706e to 0xdba9d88 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:3252 write 0x2a5e0d6 thru 0x2a64da9 (0x6cd4 bytes) 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:3253 trunc from 0xdba9d88 to 0xb791016 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:3255 punch from 0x98e315f to 0x98ee879, (0xb71a bytes) 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:3256 write 0x2734609 thru 0x2740f0c (0xc904 bytes) 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:3258 trunc from 0xb791016 to 0x214d231 2024-09-13T09:08:27.246 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.277 INFO:teuthology.orchestra.run.smithi031.stdout:3260 writesame 0xe09e114 thru 0xe0a4842 data_size 0xf(0x672f bytes) 2024-09-13T09:08:27.293 INFO:teuthology.orchestra.run.smithi031.stdout:3262 trunc from 0xe0a4843 to 0x6eb224b 2024-09-13T09:08:27.305 INFO:teuthology.orchestra.run.smithi031.stdout:3263 read 0x48fa4df thru 0x4905c7e (0xb7a0 bytes) 2024-09-13T09:08:27.305 INFO:teuthology.orchestra.run.smithi031.stdout:3264 read 0x51bc8ab thru 0x51caf70 (0xe6c6 bytes) 2024-09-13T09:08:27.305 INFO:teuthology.orchestra.run.smithi031.stdout:3266 read 0x1548a31 thru 0x154b451 (0x2a21 bytes) 2024-09-13T09:08:27.324 INFO:teuthology.orchestra.run.smithi031.stdout:3267 write 0xc00038b thru 0xc0057eb (0x5461 bytes) 2024-09-13T09:08:27.330 INFO:teuthology.orchestra.run.smithi031.stdout:3270 read 0xb36de6 thru 0xb3beef (0x510a bytes) 2024-09-13T09:08:27.331 INFO:teuthology.orchestra.run.smithi031.stdout:3271 punch from 0x8a676c3 to 0x8a748c2, (0xd1ff bytes) 2024-09-13T09:08:27.331 INFO:teuthology.orchestra.run.smithi031.stdout:3273 read 0x6a06988 thru 0x6a06c30 (0x2a9 bytes) 2024-09-13T09:08:27.331 INFO:teuthology.orchestra.run.smithi031.stdout:3274 writesame 0xa22a691 thru 0xa237c58 data_size 0x4(0xd5c8 bytes) 2024-09-13T09:08:27.344 INFO:teuthology.orchestra.run.smithi031.stdout:3276 read 0xae39bb thru 0xaebf2a (0x8570 bytes) 2024-09-13T09:08:27.344 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:27.341+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:27.346 INFO:teuthology.orchestra.run.smithi031.stdout:3277 punch from 0x65befe0 to 0x65c3204, (0x4224 bytes) 2024-09-13T09:08:27.346 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.346 INFO:teuthology.orchestra.run.smithi031.stdout:3279 read 0x4f2b347 thru 0x4f3a674 (0xf32e bytes) 2024-09-13T09:08:27.347 INFO:teuthology.orchestra.run.smithi031.stdout:3280 write 0x7a68ee5 thru 0x7a71e43 (0x8f5f bytes) 2024-09-13T09:08:27.354 INFO:teuthology.orchestra.run.smithi031.stdout:3281 read 0xa23e12a thru 0xa24494c (0x6823 bytes) 2024-09-13T09:08:27.355 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.355 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.355 INFO:teuthology.orchestra.run.smithi031.stdout:3284 write 0x103544f thru 0x103ccf7 (0x78a9 bytes) 2024-09-13T09:08:27.357 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.370 INFO:teuthology.orchestra.run.smithi031.stdout:3286 writesame 0xe038819 thru 0xe047c01 data_size 0x1(0xf3e9 bytes) 2024-09-13T09:08:27.370 INFO:teuthology.orchestra.run.smithi031.stdout:3288 read 0xc44f107 thru 0xc4554f6 (0x63f0 bytes) 2024-09-13T09:08:27.371 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.381 INFO:teuthology.orchestra.run.smithi031.stdout:3292 writesame 0xe781382 thru 0xe78e710 data_size 0x7(0xd38f bytes) 2024-09-13T09:08:27.382 INFO:teuthology.orchestra.run.smithi031.stdout:3293 punch from 0xa85718 to 0xa956ed, (0xffd5 bytes) 2024-09-13T09:08:27.395 INFO:teuthology.orchestra.run.smithi031.stdout:3294 trunc from 0xe78e711 to 0x615ab5f 2024-09-13T09:08:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:3295 punch from 0x10e15d3 to 0x10e4bd4, (0x3601 bytes) 2024-09-13T09:08:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:27.415 INFO:teuthology.orchestra.run.smithi031.stdout:3301 flatten 2024-09-13T09:08:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:3302 punch from 0xc984dc to 0xca2a54, (0xa578 bytes) 2024-09-13T09:08:27.424 INFO:teuthology.orchestra.run.smithi031.stdout:3304 clone 28 order 22 su 2097152 sc 7 2024-09-13T09:08:27.439 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:27.439 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:27.439 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:27.439 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:27.446 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:27.443+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:27.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:27.546 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:27.543+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:27.624 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:27.624 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:27.624 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:27.625 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:27.625 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:27.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:27.685 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:27.681+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:27.748 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:27.747+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:27.770 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:27.770 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:27.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:27.770 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:27.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.148770906999999, 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:27.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:27.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:08:27.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:08:27.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:27.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:08:27.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:08:27.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:08:27.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:08:27.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:08:27.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:08:27.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:08:27.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:08:27.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:08:27.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:08:27.779 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.779 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.779 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.779 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.780 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:27.780 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.148747577000002, 2024-09-13T09:08:27.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:08:27.780 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.780 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:08:27.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.782 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:27.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:27.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:08:27.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:08:27.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:08:27.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:08:27.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:27.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:08:27.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:08:27.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:08:27.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:08:27.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:08:27.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:08:27.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:08:27.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:08:27.790 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.790 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.791 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.791 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:27.791 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.148724559999998, 2024-09-13T09:08:27.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:08:27.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:08:27.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.793 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:27.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:27.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:08:27.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:08:27.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:08:27.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:08:27.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:27.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:08:27.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:08:27.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:08:27.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:08:27.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:08:27.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:08:27.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:08:27.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:08:27.801 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.801 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.801 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.802 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.802 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:27.802 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.148701430000003, 2024-09-13T09:08:27.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:08:27.802 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:08:27.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.804 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:27.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:27.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:08:27.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:08:27.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:08:27.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:08:27.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:08:27.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:08:27.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:08:27.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:08:27.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:08:27.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:08:27.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:08:27.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:08:27.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:08:27.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.812 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.146084666999997, 2024-09-13T09:08:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:08:27.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:08:27.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.815 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:27.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:27.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:08:27.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:08:27.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:08:27.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:08:27.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:08:27.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:08:27.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:08:27.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:08:27.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:08:27.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:08:27.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:08:27.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:08:27.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:08:27.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.823 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.823 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.824 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.824 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:27.824 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.146060581, 2024-09-13T09:08:27.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:08:27.825 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:08:27.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.826 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:27.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:27.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:08:27.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:08:27.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:08:27.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:08:27.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:27.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:08:27.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:08:27.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:08:27.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:08:27.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:08:27.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:08:27.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.146033928000001, 2024-09-13T09:08:27.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:08:27.839 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.146010287999999, 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.844 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.142194746999998, 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:08:27.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.142171204, 2024-09-13T09:08:27.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.142147913999999, 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:27.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:08:27.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.142124400999997, 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:08:27.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:08:27.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.142065944999999, 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.863 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:08:27.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 53.138035008999999, 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:08:27.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:08:27.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:08:27.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:27.872 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.818136711999998, 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:27.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.815764932999997, 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.877 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:08:27.881 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.815231017999999, 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.882 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.886 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.811830495000002, 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:08:27.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:08:27.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.891 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.810830688000003, 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.892 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:08:27.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 34.807660409999997, 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:27.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:27.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:08:27.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:08:27.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:27.901 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:27.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:27.903 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:27.897+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:28.003 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:28.003+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:28.078 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:28.078 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:28.078 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:28.079 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:28.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:28.140 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:08:28.141 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 0.999485 (8655022) 2024-09-13T09:08:28.141 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:28.137+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:28.204 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:28.201+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:28.224 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:28.225 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:28.225 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:28.225 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:28.225 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:28.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:28.325 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:28.321+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:28.346 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-13T09:08:28.353 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:28.355 INFO:teuthology.orchestra.run.smithi031.stdout:3306 trunc from 0x615ab5f to 0x50c36c0 2024-09-13T09:08:28.386 INFO:teuthology.orchestra.run.smithi031.stdout:3308 write 0x6e1f41d thru 0x6e294e8 (0xa0cc bytes) 2024-09-13T09:08:28.403 INFO:teuthology.orchestra.run.smithi031.stdout:3309 trunc from 0x6e294e9 to 0xd65eb38 2024-09-13T09:08:28.406 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:28.401+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:28.414 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971643870000001, 2024-09-13T09:08:28.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971619287000003, 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.420 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:28.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:08:28.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.424 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971595282999999, 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:08:28.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.9715718, 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971548392999999, 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.434 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:08:28.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.438 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971525143999997, 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.439 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.443 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971493037999998, 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:08:28.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:28.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:08:28.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:08:28.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:08:28.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:08:28.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.448 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971457420999997, 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:08:28.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:08:28.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.453 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971425441000001, 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.454 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.458 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971356161999999, 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:08:28.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.463 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971331102000001, 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971307672000002, 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.468 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:28.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:08:28.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.472 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971284289000003, 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.473 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:08:28.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.477 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971260872999999, 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:08:28.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971236949999998, 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.482 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:08:28.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.486 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971203866000003, 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:08:28.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971180066000002, 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.491 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:28.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.495 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.97115574, 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:08:28.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971123179999999, 2024-09-13T09:08:28.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 52.971083307000001, 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:28.505 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:08:28.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:28.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:08:28.509 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.510 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.510 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.510 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:28.510 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:28.510 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:28.511 INFO:teuthology.orchestra.run.smithi031.stdout:3310 writesame 0xe797e8b thru 0xe7a3a94 data_size 0x6(0xbc0a bytes) 2024-09-13T09:08:28.511 INFO:teuthology.orchestra.run.smithi031.stdout:3311 write 0x6fb3c08 thru 0x6fb472d (0xb26 bytes) 2024-09-13T09:08:28.511 INFO:teuthology.orchestra.run.smithi031.stdout:3312 writesame 0x446fdc1 thru 0x447e327 data_size 0x1(0xe567 bytes) 2024-09-13T09:08:28.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:28.512 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:28.509+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:28.526 INFO:teuthology.orchestra.run.smithi031.stdout:3313 read 0x6a348d1 thru 0x6a40783 (0xbeb3 bytes) 2024-09-13T09:08:28.526 INFO:teuthology.orchestra.run.smithi031.stdout:3314 writesame 0x9139724 thru 0x91438d4 data_size 0xb(0xa1b1 bytes) 2024-09-13T09:08:28.540 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:28.553 INFO:teuthology.orchestra.run.smithi031.stdout:3316 writesame 0xe93bf7c thru 0xe94813d data_size 0x15(0xc1c2 bytes) 2024-09-13T09:08:28.554 INFO:teuthology.orchestra.run.smithi031.stdout:3317 read 0x5a02943 thru 0x5a07dbe (0x547c bytes) 2024-09-13T09:08:28.554 INFO:teuthology.orchestra.run.smithi031.stdout:3319 read 0x5c123e7 thru 0x5c1245a (0x74 bytes) 2024-09-13T09:08:28.554 INFO:teuthology.orchestra.run.smithi031.stdout:3322 write 0x983545c thru 0x9838979 (0x351e bytes) 2024-09-13T09:08:28.561 INFO:teuthology.orchestra.run.smithi031.stdout:3324 write 0xb38ef01 thru 0xb39eb87 (0xfc87 bytes) 2024-09-13T09:08:28.569 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:28.569 INFO:teuthology.orchestra.run.smithi031.stdout:3326 writesame 0x7f3885f thru 0x7f3c02e data_size 0x4c(0x37d0 bytes) 2024-09-13T09:08:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:3327 trunc from 0xe94813e to 0xe848d71 2024-09-13T09:08:28.595 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:28.595 INFO:teuthology.orchestra.run.smithi031.stdout:3331 writesame 0xa90490d thru 0xa91371e data_size 0x1f(0xee12 bytes) 2024-09-13T09:08:28.605 INFO:teuthology.orchestra.run.smithi031.stdout:3333 clone 29 order 20 su 4096 sc 11 2024-09-13T09:08:28.612 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:28.611+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:28.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:28.687 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:28.687 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:28.688 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:28.688 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:28.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:28.713 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:28.711+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:28.811 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:28.812 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:28.814 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:28.811+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:28.828 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:28.828 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:28.828 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:28.828 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:28.828 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:28.829 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:28.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:28.844 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:28.860 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:28.860 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:28.878 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:28.878 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:28.896 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:28.897 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:28.915 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:28.909+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:28.915 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:28.916 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:28.937 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:28.937 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:28.970 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:28.970 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:28.985 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:29.047 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:29.047+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:29.117 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:29.113+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:29.128 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:29.128 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:29.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:29.218 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:29.213+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:29.271 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone28 from 0xe848d71 (overlap 0x50c36c0) to 0x204e2b3 2024-09-13T09:08:29.319 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:29.313+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:29.320 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:29.336 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:29.400 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-13T09:08:29.419 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:29.419+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:29.463 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:29.463 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:29.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:29.521 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:29.517+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:29.622 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:29.622 INFO:teuthology.orchestra.run.smithi031.stdout:3335 write 0x31883c0 thru 0x3195fd4 (0xdc15 bytes) 2024-09-13T09:08:29.622 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:29.617+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:29.654 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:29.654 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:29.674 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:29.723 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:29.719+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:29.824 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:29.817+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:29.835 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:29.835 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:29.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:29.943 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:29.943+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:30.014 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:30.014 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:30.026 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:30.023+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:30.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:30.127 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:30.121+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:30.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:08:30.193 INFO:teuthology.orchestra.run.smithi031.stdout:3336 write 0x9b02b64 thru 0x9b05ef9 (0x3396 bytes) 2024-09-13T09:08:30.206 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:30.207 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:30.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:30.227 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:30.221+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:30.243 INFO:teuthology.orchestra.run.smithi031.stdout:3338 read 0xdd2bed thru 0xdd930c (0x6720 bytes) 2024-09-13T09:08:30.248 INFO:teuthology.orchestra.run.smithi031.stdout:3339 read 0x7c2539c thru 0x7c2cb88 (0x77ed bytes) 2024-09-13T09:08:30.248 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:30.248 INFO:teuthology.orchestra.run.smithi031.stdout:3341 read 0x57b3e57 thru 0x57b8eec (0x5096 bytes) 2024-09-13T09:08:30.249 INFO:teuthology.orchestra.run.smithi031.stdout:3344 write 0xc0fd93b thru 0xc10272d (0x4df3 bytes) 2024-09-13T09:08:30.328 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:30.327+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:30.370 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:30.370 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:30.382 INFO:teuthology.orchestra.run.smithi031.stdout:3345 read 0x9fc825a thru 0x9fd5acf (0xd876 bytes) 2024-09-13T09:08:30.384 INFO:teuthology.orchestra.run.smithi031.stdout:3346 read 0x99d7500 thru 0x99e0ca4 (0x97a5 bytes) 2024-09-13T09:08:30.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:30.388 INFO:teuthology.orchestra.run.smithi031.stdout:3347 punch from 0x7f0d1ff to 0x7f11fa3, (0x4da4 bytes) 2024-09-13T09:08:30.388 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:30.395 INFO:teuthology.orchestra.run.smithi031.stdout:3352 trunc from 0xe848d71 to 0x967a895 2024-09-13T09:08:30.429 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:30.425+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:30.530 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:30.527+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:30.532 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:30.532 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:30.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:30.635 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:30.635+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:30.700 INFO:teuthology.orchestra.run.smithi031.stdout:3353 write 0xd4f69b1 thru 0xd502c97 (0xc2e7 bytes) 2024-09-13T09:08:30.724 INFO:teuthology.orchestra.run.smithi031.stdout:3354 writesame 0x436608 thru 0x43ce3c data_size 0x25(0x6835 bytes) 2024-09-13T09:08:30.728 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:30.728 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:30.731 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:30.731+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:30.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:30.843 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:30.843+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:30.933 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:30.931+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:30.941 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:30.941 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:30.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:30.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:30.986 INFO:teuthology.orchestra.run.smithi031.stdout:3355 writesame 0xca69eae thru 0xca77798 data_size 0x1(0xd8eb bytes) 2024-09-13T09:08:31.028 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:31.034 INFO:teuthology.orchestra.run.smithi031.stdout:3359 write 0xd65e2e8 thru 0xd669611 (0xb32a bytes) 2024-09-13T09:08:31.035 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:31.035+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:31.039 INFO:teuthology.orchestra.run.smithi031.stdout:3360 write 0x4edebd7 thru 0x4ee4faa (0x63d4 bytes) 2024-09-13T09:08:31.112 INFO:teuthology.orchestra.run.smithi031.stdout:3361 read 0x9ebb363 thru 0x9ec2d75 (0x7a13 bytes) 2024-09-13T09:08:31.113 INFO:teuthology.orchestra.run.smithi031.stdout:3362 punch from 0x23bef17 to 0x23cc1a1, (0xd28a bytes) 2024-09-13T09:08:31.113 INFO:teuthology.orchestra.run.smithi031.stdout:3363 trunc from 0xd669612 to 0xd2e3ac9 2024-09-13T09:08:31.135 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:31.129+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:31.142 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:31.142 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:31.157 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:08:31.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:31.165 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:31.165 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:31.182 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:31.182 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:31.198 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:31.198 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:31.214 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:31.214 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:31.233 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:31.233 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:31.236 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:31.229+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:31.252 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:31.252 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:31.295 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:31.295 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:31.295 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:31.295 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:31.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:31.350 INFO:teuthology.orchestra.run.smithi031.stdout:3364 trunc from 0xd2e3ac9 to 0x95662f4 2024-09-13T09:08:31.353 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:31.349+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:31.387 INFO:teuthology.orchestra.run.smithi031.stdout:3365 trunc from 0x95662f4 to 0x8742f4d 2024-09-13T09:08:31.438 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:31.433+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:31.468 INFO:teuthology.orchestra.run.smithi031.stdout:3366 trunc from 0x8742f4d to 0x20afe7 2024-09-13T09:08:31.482 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:31.482 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:31.494 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:31.494 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:31.494 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:31.494 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:31.495 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:31.506 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:31.506 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:31.513 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:31.575 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:31.575+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:31.641 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:31.639+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:31.646 INFO:teuthology.orchestra.run.smithi031.stdout:3367 writesame 0x16ff261 thru 0x170012c data_size 0x2(0xecc bytes) 2024-09-13T09:08:31.651 INFO:teuthology.orchestra.run.smithi031.stdout:3368 trunc from 0x170012d to 0x31c5647 2024-09-13T09:08:31.661 INFO:teuthology.orchestra.run.smithi031.stdout:3369 read 0x89e3a3 thru 0x8ab5de (0xd23c bytes) 2024-09-13T09:08:31.663 INFO:teuthology.orchestra.run.smithi031.stdout:3370 read 0x16a329a thru 0x16b034d (0xd0b4 bytes) 2024-09-13T09:08:31.678 INFO:teuthology.orchestra.run.smithi031.stdout:3371 write 0x43b9d4e thru 0x43c01ed (0x64a0 bytes) 2024-09-13T09:08:31.694 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:31.694 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:31.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:31.694 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.668105939, 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.695 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:08:31.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.699 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.668058605999999, 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:31.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:08:31.703 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.668001069999999, 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:08:31.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:31.708 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.667949583999999, 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:31.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:08:31.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:08:31.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.712 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.667902642000001, 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:08:31.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:08:31.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.667855248999999, 2024-09-13T09:08:31.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.721 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.667807973999999, 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:08:31.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:08:31.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:31.726 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.667761773999999, 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:08:31.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:08:31.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:31.731 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.667697769, 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:08:31.735 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.66522668, 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.736 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:08:31.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.740 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.665170958000001, 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:08:31.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.665121055, 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:08:31.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.749 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.665073999000001, 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:08:31.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.665025516, 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.754 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:08:31.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:08:31.758 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.664960251, 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:08:31.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.664913335000001, 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:08:31.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:08:31.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:08:31.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.664825886999999, 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:08:31.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:08:31.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:08:31.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:31.775 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.663364038000001, 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:31.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:08:31.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:08:31.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:31.780 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.663285951999999, 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:08:31.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.663233862999999, 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:08:31.785 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:31.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:08:31.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:31.790 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:31.790 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:3372 flatten 2024-09-13T09:08:31.791 INFO:teuthology.orchestra.run.smithi031.stdout:3374 write 0x2784fdc thru 0x278502d (0x52 bytes) 2024-09-13T09:08:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:3377 writesame 0x7d73212 thru 0x7d7faa5 data_size 0x42(0xc894 bytes) 2024-09-13T09:08:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:3378 read 0x3a8441b thru 0x3a8f25b (0xae41 bytes) 2024-09-13T09:08:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:3379 read 0x2a62a8d thru 0x2a6a975 (0x7ee9 bytes) 2024-09-13T09:08:31.792 INFO:teuthology.orchestra.run.smithi031.stdout:3381 writesame 0x1b0a1ce thru 0x1b13091 data_size 0x2(0x8ec4 bytes) 2024-09-13T09:08:31.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:31.793 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:31.785+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:3382 read 0xedd682 thru 0xeec7e3 (0xf162 bytes) 2024-09-13T09:08:31.794 INFO:teuthology.orchestra.run.smithi031.stdout:3383 read 0x2f9d399 thru 0x2fa5141 (0x7da9 bytes) 2024-09-13T09:08:31.814 INFO:teuthology.orchestra.run.smithi031.stdout:3384 writesame 0x8f856aa thru 0x8f8807b data_size 0x2(0x29d2 bytes) 2024-09-13T09:08:31.815 INFO:teuthology.orchestra.run.smithi031.stdout:3385 trunc from 0x8f8807c to 0x88f5f29 2024-09-13T09:08:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:3386 punch from 0x12faf59 to 0x130a915, (0xf9bc bytes) 2024-09-13T09:08:31.834 INFO:teuthology.orchestra.run.smithi031.stdout:3387 writesame 0x4fd76e0 thru 0x4fe6fd4 data_size 0x17(0xf8f5 bytes) 2024-09-13T09:08:31.866 INFO:teuthology.orchestra.run.smithi031.stdout:3388 write 0x2cca91d thru 0x2cd61a7 (0xb88b bytes) 2024-09-13T09:08:31.892 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:31.891+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:31.909 INFO:teuthology.orchestra.run.smithi031.stdout:3389 trunc from 0x88f5f29 to 0xdb4a663 2024-09-13T09:08:31.921 INFO:teuthology.orchestra.run.smithi031.stdout:3390 punch from 0xae158c3 to 0xae18463, (0x2ba0 bytes) 2024-09-13T09:08:31.932 INFO:teuthology.orchestra.run.smithi031.stdout:3393 trunc from 0xdb4a663 to 0x90f9aa2 2024-09-13T09:08:31.948 INFO:teuthology.orchestra.run.smithi031.stdout:3396 read 0x7178150 thru 0x717eb55 (0x6a06 bytes) 2024-09-13T09:08:31.949 INFO:teuthology.orchestra.run.smithi031.stdout:3398 read 0x6bdca99 thru 0x6bea845 (0xddad bytes) 2024-09-13T09:08:31.949 INFO:teuthology.orchestra.run.smithi031.stdout:3399 punch from 0x16420cb to 0x164af72, (0x8ea7 bytes) 2024-09-13T09:08:31.969 INFO:teuthology.orchestra.run.smithi031.stdout:3400 write 0xa7b013f thru 0xa7bc969 (0xc82b bytes) 2024-09-13T09:08:31.969 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:31.969 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:31.970 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:31.970 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:31.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:31.994 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:31.991+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:32.000 INFO:teuthology.orchestra.run.smithi031.stdout:3402 read 0x99790cf thru 0x9987a94 (0xe9c6 bytes) 2024-09-13T09:08:32.019 INFO:teuthology.orchestra.run.smithi031.stdout:3404 trunc from 0xa7bc96a to 0x3c47900 2024-09-13T09:08:32.051 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.052 INFO:teuthology.orchestra.run.smithi031.stdout:3406 punch from 0x1c78c22 to 0x1c7d435, (0x4813 bytes) 2024-09-13T09:08:32.052 INFO:teuthology.orchestra.run.smithi031.stdout:3407 writesame 0x3030517 thru 0x303d496 data_size 0x80(0xcf80 bytes) 2024-09-13T09:08:32.078 INFO:teuthology.orchestra.run.smithi031.stdout:3408 writesame 0x6d5c129 thru 0x6d5f1b2 data_size 0x26(0x308a bytes) 2024-09-13T09:08:32.079 INFO:teuthology.orchestra.run.smithi031.stdout:3410 read 0xf02693 thru 0xf02f27 (0x895 bytes) 2024-09-13T09:08:32.094 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:32.089+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:32.098 INFO:teuthology.orchestra.run.smithi031.stdout:3413 writesame 0x9f4e3bf thru 0x9f521e2 data_size 0x4(0x3e24 bytes) 2024-09-13T09:08:32.098 INFO:teuthology.orchestra.run.smithi031.stdout:3414 read 0x1a0cd03 thru 0x1a134a0 (0x679e bytes) 2024-09-13T09:08:32.099 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.113 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:32.113 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:32.113 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:32.113 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:32.113 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:32.121 INFO:teuthology.orchestra.run.smithi031.stdout:3418 writesame 0xdae98cd thru 0xdaf1188 data_size 0x2(0x78bc bytes) 2024-09-13T09:08:32.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:32.155 INFO:teuthology.orchestra.run.smithi031.stdout:3419 trunc from 0xdaf1189 to 0x1623458 2024-09-13T09:08:32.208 INFO:teuthology.orchestra.run.smithi031.stdout:3420 writesame 0x4d0a74e thru 0x4d1254b data_size 0x1(0x7dfe bytes) 2024-09-13T09:08:32.208 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.221 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:32.217+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:32.232 INFO:teuthology.orchestra.run.smithi031.stdout:3424 write 0x9e7f44b thru 0x9e843f6 (0x4fac bytes) 2024-09-13T09:08:32.272 INFO:teuthology.orchestra.run.smithi031.stdout:3425 writesame 0xa9e9f0d thru 0xa9f82b7 data_size 0x1(0xe3ab bytes) 2024-09-13T09:08:32.280 INFO:teuthology.orchestra.run.smithi031.stdout:3426 trunc from 0xa9f82b8 to 0x803fb12 2024-09-13T09:08:32.296 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:32.295+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:32.303 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:32.303 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:32.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:32.303 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:32.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.303 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.303 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.683170740999998, 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:08:32.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:08:32.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.683147411, 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:32.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.312 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.683124394000004, 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:08:32.316 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.683101264000001, 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.317 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:08:32.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.321 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.680484501000002, 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:08:32.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:08:32.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.680460414999999, 2024-09-13T09:08:32.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:08:32.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:08:32.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.680433762, 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:08:32.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:08:32.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.335 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.680410121999998, 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:32.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:08:32.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.676594581000003, 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.340 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:08:32.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:32.344 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.676571037999999, 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.676547747999997, 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.349 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:08:32.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:08:32.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.676524235000002, 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.353 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.676465778999997, 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:08:32.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:08:32.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.361 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.672434842999998, 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.362 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:08:32.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.352536546000003, 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:08:32.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:08:32.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.371 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.350164767000003, 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:32.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:08:32.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:32.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.376 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.349630851999997, 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.377 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:08:32.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:32.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.346230329000001, 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:08:32.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:08:32.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.345230522000001, 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:08:32.386 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:08:32.390 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 39.342060244000002, 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.391 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:08:32.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:32.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.396 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.396 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:32.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:32.398 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.398 INFO:teuthology.orchestra.run.smithi031.stdout:3428 punch from 0x6c2a5ec to 0x6c31e1d, (0x7831 bytes) 2024-09-13T09:08:32.398 INFO:teuthology.orchestra.run.smithi031.stdout:3429 punch from 0x3f46132 to 0x3f48e86, (0x2d54 bytes) 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:3431 writesame 0xe0f7736 thru 0xe105a00 data_size 0x3(0xe2cb bytes) 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:3432 punch from 0x8ca43df to 0x8ca4864, (0x485 bytes) 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:3433 writesame 0x32f69e2 thru 0x32fe490 data_size 0x1d(0x7aaf bytes) 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:3434 read 0x788ec80 thru 0x789a467 (0xb7e8 bytes) 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:3439 read 0x313c86 thru 0x31eaa8 (0xae23 bytes) 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:3442 read 0xcc73e02 thru 0xcc786de (0x48dd bytes) 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:3443 punch from 0xba83f4a to 0xba8cb2e, (0x8be4 bytes) 2024-09-13T09:08:32.399 INFO:teuthology.orchestra.run.smithi031.stdout:3444 trunc from 0xe105a01 to 0x798fd1c 2024-09-13T09:08:32.399 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:32.395+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:32.402 INFO:teuthology.orchestra.run.smithi031.stdout:3445 read 0x5f07a04 thru 0x5f0bc10 (0x420d bytes) 2024-09-13T09:08:32.402 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.402 INFO:teuthology.orchestra.run.smithi031.stdout:3450 punch from 0x74c272f to 0x74d136a, (0xec3b bytes) 2024-09-13T09:08:32.402 INFO:teuthology.orchestra.run.smithi031.stdout:3453 read 0x6c5b95a thru 0x6c647b8 (0x8e5f bytes) 2024-09-13T09:08:32.402 INFO:teuthology.orchestra.run.smithi031.stdout:3454 read 0x43e9a5e thru 0x43f1451 (0x79f4 bytes) 2024-09-13T09:08:32.423 INFO:teuthology.orchestra.run.smithi031.stdout:3455 writesame 0x92b8b7c thru 0x92c8b0a data_size 0x1(0xff8f bytes) 2024-09-13T09:08:32.423 INFO:teuthology.orchestra.run.smithi031.stdout:3456 read 0x49815a7 thru 0x4986fb9 (0x5a13 bytes) 2024-09-13T09:08:32.426 INFO:teuthology.orchestra.run.smithi031.stdout:3457 trunc from 0x92c8b0b to 0x8127b08 2024-09-13T09:08:32.451 INFO:teuthology.orchestra.run.smithi031.stdout:3459 trunc from 0x8127b08 to 0xb2e9414 2024-09-13T09:08:32.464 INFO:teuthology.orchestra.run.smithi031.stdout:3461 write 0xa6d37d8 thru 0xa6def66 (0xb78f bytes) 2024-09-13T09:08:32.497 INFO:teuthology.orchestra.run.smithi031.stdout:3462 write 0x3070fe0 thru 0x30791c7 (0x81e8 bytes) 2024-09-13T09:08:32.498 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:32.495+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:3464 trunc from 0xb2e9414 to 0xd05044f 2024-09-13T09:08:32.526 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.526 INFO:teuthology.orchestra.run.smithi031.stdout:3467 writesame 0x9a52489 thru 0x9a57c47 data_size 0x1(0x57bf bytes) 2024-09-13T09:08:32.570 INFO:teuthology.orchestra.run.smithi031.stdout:3468 write 0x8544578 thru 0x854fc4f (0xb6d8 bytes) 2024-09-13T09:08:32.581 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:32.582 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:32.582 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:32.582 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:32.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:32.598 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:32.593+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:32.604 INFO:teuthology.orchestra.run.smithi031.stdout:3469 punch from 0x40e5c9a to 0x40f16dd, (0xba43 bytes) 2024-09-13T09:08:32.604 INFO:teuthology.orchestra.run.smithi031.stdout:3470 read 0xba32da2 thru 0xba422b7 (0xf516 bytes) 2024-09-13T09:08:32.605 INFO:teuthology.orchestra.run.smithi031.stdout:3472 read 0x6074375 thru 0x6077672 (0x32fe bytes) 2024-09-13T09:08:32.605 INFO:teuthology.orchestra.run.smithi031.stdout:3474 write 0x4d50e8f thru 0x4d5a972 (0x9ae4 bytes) 2024-09-13T09:08:32.632 INFO:teuthology.orchestra.run.smithi031.stdout:3475 punch from 0x9cb5148 to 0x9cb926c, (0x4124 bytes) 2024-09-13T09:08:32.632 INFO:teuthology.orchestra.run.smithi031.stdout:3476 read 0xab6f0d8 thru 0xab75bcb (0x6af4 bytes) 2024-09-13T09:08:32.633 INFO:teuthology.orchestra.run.smithi031.stdout:3478 read 0xabee08d thru 0xabf10eb (0x305f bytes) 2024-09-13T09:08:32.648 INFO:teuthology.orchestra.run.smithi031.stdout:3479 write 0xe30a2e9 thru 0xe3164d2 (0xc1ea bytes) 2024-09-13T09:08:32.673 INFO:teuthology.orchestra.run.smithi031.stdout:3480 punch from 0xccf02f5 to 0xccf5859, (0x5564 bytes) 2024-09-13T09:08:32.674 INFO:teuthology.orchestra.run.smithi031.stdout:3481 writesame 0x674e7e7 thru 0x67504a1 data_size 0x1(0x1cbb bytes) 2024-09-13T09:08:32.685 INFO:teuthology.orchestra.run.smithi031.stdout:3482 writesame 0xad264d0 thru 0xad2eb00 data_size 0x21(0x8631 bytes) 2024-09-13T09:08:32.690 INFO:teuthology.orchestra.run.smithi031.stdout:3484 read 0x3e8894f thru 0x3e8febf (0x7571 bytes) 2024-09-13T09:08:32.690 INFO:teuthology.orchestra.run.smithi031.stdout:3485 write 0x381c601 thru 0x381e446 (0x1e46 bytes) 2024-09-13T09:08:32.699 INFO:teuthology.orchestra.run.smithi031.stdout:3487 punch from 0x9a7a3eb to 0x9a88ad1, (0xe6e6 bytes) 2024-09-13T09:08:32.699 INFO:teuthology.orchestra.run.smithi031.stdout:3488 read 0xb84c2b9 thru 0xb84cd42 (0xa8a bytes) 2024-09-13T09:08:32.699 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.699 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.700 INFO:teuthology.orchestra.run.smithi031.stdout:3492 read 0xd050d9e thru 0xd050e12 (0x75 bytes) 2024-09-13T09:08:32.700 INFO:teuthology.orchestra.run.smithi031.stdout:3495 write 0x185455d thru 0x185ea08 (0xa4ac bytes) 2024-09-13T09:08:32.700 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:32.693+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:32.728 INFO:teuthology.orchestra.run.smithi031.stdout:3496 trunc from 0xe3164d3 to 0x4cbaf79 2024-09-13T09:08:32.736 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:32.736 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:32.736 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:32.736 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:32.736 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:32.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:32.788 INFO:teuthology.orchestra.run.smithi031.stdout:3497 write 0xa270de8 thru 0xa27d84a (0xca63 bytes) 2024-09-13T09:08:32.800 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:32.797+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:32.815 INFO:teuthology.orchestra.run.smithi031.stdout:3499 punch from 0x1ca122b to 0x1caf966, (0xe73b bytes) 2024-09-13T09:08:32.815 INFO:teuthology.orchestra.run.smithi031.stdout:3500 read 0x3fe0681 thru 0x3fe5d4d (0x56cd bytes) 2024-09-13T09:08:32.838 INFO:teuthology.orchestra.run.smithi031.stdout:3501 write 0xe61d7ab thru 0xe628323 (0xab79 bytes) 2024-09-13T09:08:32.867 INFO:teuthology.orchestra.run.smithi031.stdout:3503 writesame 0xcb1b8c0 thru 0xcb20efb data_size 0x2(0x563c bytes) 2024-09-13T09:08:32.888 INFO:teuthology.orchestra.run.smithi031.stdout:3504 write 0x9aec8f6 thru 0x9af68e0 (0x9feb bytes) 2024-09-13T09:08:32.895 INFO:teuthology.orchestra.run.smithi031.stdout:3505 read 0x855c783 thru 0x8563865 (0x70e3 bytes) 2024-09-13T09:08:32.895 INFO:teuthology.orchestra.run.smithi031.stdout:3506 write 0xb94d8c8 thru 0xb954c47 (0x7380 bytes) 2024-09-13T09:08:32.901 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:32.899+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:32.915 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:32.915 INFO:teuthology.orchestra.run.smithi031.stdout:3508 read 0x20d60c2 thru 0x20db4ba (0x53f9 bytes) 2024-09-13T09:08:32.916 INFO:teuthology.orchestra.run.smithi031.stdout:3509 write 0x4acb25 thru 0x4b91e0 (0xc6bc bytes) 2024-09-13T09:08:32.937 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:32.937 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494345590999998, 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.942 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494321008, 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:08:32.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.947 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494297004000003, 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:08:32.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.952 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494273520999997, 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:08:32.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:08:32.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:08:32.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:08:32.957 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494250114000003, 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.958 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:08:32.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494226865000002, 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:08:32.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:08:32.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:08:32.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:08:32.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494194759000003, 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.968 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:08:32.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494159142000001, 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:08:32.973 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:08:32.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.977 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494127161999998, 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:08:32.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494057882999996, 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:08:32.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:08:32.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:08:32.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:32.987 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494032822999998, 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.991 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.494009392999999, 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:08:32.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:08:32.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.996 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.49398601, 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:08:32.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.493962594000003, 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.001 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:08:33.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.005 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.493938671000002, 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.006 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:33.010 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.493905587, 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:08:33.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.014 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.493881786999999, 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.015 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:08:33.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.493857460999997, 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:08:33.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:08:33.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.493824900999996, 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.024 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:08:33.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 57.493785027999998, 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:08:33.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:33.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:08:33.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:08:33.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:08:33.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.034 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:33.035 INFO:teuthology.orchestra.run.smithi031.stdout:3512 trunc from 0xe628324 to 0x63be94b 2024-09-13T09:08:33.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:33.036 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:33.029+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:33.058 INFO:teuthology.orchestra.run.smithi031.stdout:3513 trunc from 0x63be94b to 0x5bfd1e0 2024-09-13T09:08:33.071 INFO:teuthology.orchestra.run.smithi031.stdout:3514 read 0x46b7340 thru 0x46bacd5 (0x3996 bytes) 2024-09-13T09:08:33.092 INFO:teuthology.orchestra.run.smithi031.stdout:3516 writesame 0x846d003 thru 0x84773c9 data_size 0x1(0xa3c7 bytes) 2024-09-13T09:08:33.092 INFO:teuthology.orchestra.run.smithi031.stdout:3517 read 0x18b107c thru 0x18b4723 (0x36a8 bytes) 2024-09-13T09:08:33.107 INFO:teuthology.orchestra.run.smithi031.stdout:3518 write 0x93de0f4 thru 0x93e36fa (0x5607 bytes) 2024-09-13T09:08:33.127 INFO:teuthology.orchestra.run.smithi031.stdout:3519 read 0x58c3686 thru 0x58d05dd (0xcf58 bytes) 2024-09-13T09:08:33.137 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:33.135+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:33.151 INFO:teuthology.orchestra.run.smithi031.stdout:3520 trunc from 0x93e36fb to 0xd5af63 2024-09-13T09:08:33.170 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:08:33.170 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:08:33.170 INFO:tasks.thrashosds.thrasher:Reweighting osd 0 to 0.24778157884346358 2024-09-13T09:08:33.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 0.24778157884346358 2024-09-13T09:08:33.177 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:08:33.179 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:33.179 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:33.180 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:33.180 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:33.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:33.204 INFO:teuthology.orchestra.run.smithi031.stdout:3521 writesame 0x1d41a1e thru 0x1d44bc8 data_size 0x1(0x31ab bytes) 2024-09-13T09:08:33.217 INFO:teuthology.orchestra.run.smithi031.stdout:3524 writesame 0x426a8db thru 0x4272c3b data_size 0x25(0x8361 bytes) 2024-09-13T09:08:33.218 INFO:teuthology.orchestra.run.smithi031.stdout:3525 write 0x3c4ec61 thru 0x3c513b1 (0x2751 bytes) 2024-09-13T09:08:33.227 INFO:teuthology.orchestra.run.smithi031.stdout:3526 read 0x31ee96 thru 0x3274e7 (0x8652 bytes) 2024-09-13T09:08:33.235 INFO:teuthology.orchestra.run.smithi031.stdout:3527 write 0x458ed9d thru 0x459cbe0 (0xde44 bytes) 2024-09-13T09:08:33.237 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:33.235+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:33.260 INFO:teuthology.orchestra.run.smithi031.stdout:3528 read 0x3ba4c36 thru 0x3bb1659 (0xca24 bytes) 2024-09-13T09:08:33.260 INFO:teuthology.orchestra.run.smithi031.stdout:3530 write 0x31fd793 thru 0x320aa1c (0xd28a bytes) 2024-09-13T09:08:33.278 INFO:teuthology.orchestra.run.smithi031.stdout:3532 read 0x412a70f thru 0x41349d3 (0xa2c5 bytes) 2024-09-13T09:08:33.279 INFO:teuthology.orchestra.run.smithi031.stdout:3533 read 0x170a945 thru 0x1716736 (0xbdf2 bytes) 2024-09-13T09:08:33.279 INFO:teuthology.orchestra.run.smithi031.stdout:3535 writesame 0x37dd3d7 thru 0x37ebcd3 data_size 0x5(0xe8fd bytes) 2024-09-13T09:08:33.296 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.297 INFO:teuthology.orchestra.run.smithi031.stdout:3539 read 0x153c3d0 thru 0x1547b5e (0xb78f bytes) 2024-09-13T09:08:33.297 INFO:teuthology.orchestra.run.smithi031.stdout:3541 punch from 0x1483dea to 0x1486525, (0x273b bytes) 2024-09-13T09:08:33.320 INFO:teuthology.orchestra.run.smithi031.stdout:3543 writesame 0xdd9eac3 thru 0xddae4ee data_size 0x1b(0xfa2c bytes) 2024-09-13T09:08:33.323 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:33.323 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:33.324 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:33.324 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:33.331 INFO:teuthology.orchestra.run.smithi031.stdout:3544 trunc from 0xddae4ef to 0x6df30e0 2024-09-13T09:08:33.338 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:33.335+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:33.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:33.342 INFO:teuthology.orchestra.run.smithi031.stdout:3547 read 0x923ff thru 0xa0f1b (0xeb1d bytes) 2024-09-13T09:08:33.344 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.344 INFO:teuthology.orchestra.run.smithi031.stdout:3551 read 0x4fc083f thru 0x4fc39a4 (0x3166 bytes) 2024-09-13T09:08:33.344 INFO:teuthology.orchestra.run.smithi031.stdout:3552 read 0x60c68ec thru 0x60d0747 (0x9e5c bytes) 2024-09-13T09:08:33.344 INFO:teuthology.orchestra.run.smithi031.stdout:3553 punch from 0x269cd11 to 0x26a0485, (0x3774 bytes) 2024-09-13T09:08:33.345 INFO:teuthology.orchestra.run.smithi031.stdout:3554 read 0x3177b13 thru 0x317ef7f (0x746d bytes) 2024-09-13T09:08:33.345 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.366 INFO:teuthology.orchestra.run.smithi031.stdout:3556 writesame 0xe6609fa thru 0xe6613ab data_size 0x2(0x9b2 bytes) 2024-09-13T09:08:33.366 INFO:teuthology.orchestra.run.smithi031.stdout:3557 write 0x779f334 thru 0x77acbea (0xd8b7 bytes) 2024-09-13T09:08:33.385 INFO:teuthology.orchestra.run.smithi031.stdout:3558 trunc from 0xe6613ac to 0xdf1577e 2024-09-13T09:08:33.393 INFO:teuthology.orchestra.run.smithi031.stdout:3559 read 0x77301e6 thru 0x7730b74 (0x98f bytes) 2024-09-13T09:08:33.394 INFO:teuthology.orchestra.run.smithi031.stdout:3560 read 0x13c5777 thru 0x13d4795 (0xf01f bytes) 2024-09-13T09:08:33.394 INFO:teuthology.orchestra.run.smithi031.stdout:3562 read 0x57685ba thru 0x57765f7 (0xe03e bytes) 2024-09-13T09:08:33.394 INFO:teuthology.orchestra.run.smithi031.stdout:3563 writesame 0x7491f26 thru 0x7497a7d data_size 0x8(0x5b58 bytes) 2024-09-13T09:08:33.399 INFO:teuthology.orchestra.run.smithi031.stdout:3564 read 0x75659fb thru 0x7568d32 (0x3338 bytes) 2024-09-13T09:08:33.400 INFO:teuthology.orchestra.run.smithi031.stdout:3565 punch from 0x89ae705 to 0x89b9846, (0xb141 bytes) 2024-09-13T09:08:33.400 INFO:teuthology.orchestra.run.smithi031.stdout:3566 writesame 0x13d8647 thru 0x13e256b data_size 0x1(0x9f25 bytes) 2024-09-13T09:08:33.432 INFO:teuthology.orchestra.run.smithi031.stdout:3568 read 0xb511892 thru 0xb51d631 (0xbda0 bytes) 2024-09-13T09:08:33.432 INFO:teuthology.orchestra.run.smithi031.stdout:3569 write 0x7d00c1d thru 0x7d0ef43 (0xe327 bytes) 2024-09-13T09:08:33.439 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:33.433+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:33.451 INFO:teuthology.orchestra.run.smithi031.stdout:3570 read 0x2b357c9 thru 0x2b429d8 (0xd210 bytes) 2024-09-13T09:08:33.451 INFO:teuthology.orchestra.run.smithi031.stdout:3572 read 0xbf20417 thru 0xbf2262c (0x2216 bytes) 2024-09-13T09:08:33.451 INFO:teuthology.orchestra.run.smithi031.stdout:3573 write 0x4555c6d thru 0x4563c68 (0xdffc bytes) 2024-09-13T09:08:33.467 INFO:teuthology.orchestra.run.smithi031.stdout:3574 trunc from 0xdf1577e to 0x6f2dc06 2024-09-13T09:08:33.469 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.24433492, 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244304069999998, 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:08:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244277902999997, 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:08:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244238428999999, 2024-09-13T09:08:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:08:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:08:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244203874999997, 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244171518999998, 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244139248000003, 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:08:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244105212000001, 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:08:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:08:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244071947999998, 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.244039708000003, 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.235059622000001, 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:08:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:08:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:08:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.235018742000001, 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.234978474000002, 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:08:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:08:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.234931431, 2024-09-13T09:08:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.234885869000003, 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:08:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.234795701000003, 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:08:33.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:08:33.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.234749147000002, 2024-09-13T09:08:33.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:08:33.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.233108223000002, 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.548 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.549 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.551 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.552 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.233042417, 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.553 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.554 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:08:33.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.556 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.557 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.55003477199999995, 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.558 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:33.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:33.560 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:33.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:08:33.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:08:33.564 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.564 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.564 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.564 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:33.564 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:33.564 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:33.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:33.567 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:33.561+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:33.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3577 writesame 0x9257ce8 thru 0x9262c19 data_size 0x41(0xaf32 bytes) 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3579 trunc from 0x9262c1a to 0xe930111 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3580 read 0x66c7b27 thru 0x66d543b (0xd915 bytes) 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3581 read 0x1056544 thru 0x1059910 (0x33cd bytes) 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3582 read 0x51025ce thru 0x5109b19 (0x754c bytes) 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3583 punch from 0x8819414 to 0x881d370, (0x3f5c bytes) 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3585 write 0xecd8ae3 thru 0xecdc4e4 (0x3a02 bytes) 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3587 read 0x115f8e1 thru 0x116c641 (0xcd61 bytes) 2024-09-13T09:08:33.569 INFO:teuthology.orchestra.run.smithi031.stdout:3589 punch from 0x6a33b4d to 0x6a354c7, (0x197a bytes) 2024-09-13T09:08:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:3593 read 0x6e9f1c7 thru 0x6eaa9f1 (0xb82b bytes) 2024-09-13T09:08:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:3594 writesame 0xbc786b8 thru 0xbc80ce7 data_size 0x71(0x8630 bytes) 2024-09-13T09:08:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:3595 writesame 0x42e3e4c thru 0x42e4dd2 data_size 0x19(0xf87 bytes) 2024-09-13T09:08:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:3596 write 0x3e8ebb5 thru 0x3e99424 (0xa870 bytes) 2024-09-13T09:08:33.570 INFO:teuthology.orchestra.run.smithi031.stdout:3597 trunc from 0xecdc4e5 to 0x92af9e2 2024-09-13T09:08:33.587 INFO:teuthology.orchestra.run.smithi031.stdout:3598 write 0x5a2c4d thru 0x5a7134 (0x44e8 bytes) 2024-09-13T09:08:33.605 INFO:teuthology.orchestra.run.smithi031.stdout:3599 writesame 0xeb9f564 thru 0xebad8be data_size 0x1d(0xe35b bytes) 2024-09-13T09:08:33.605 INFO:teuthology.orchestra.run.smithi031.stdout:3601 punch from 0x9705cb8 to 0x9710f54, (0xb29c bytes) 2024-09-13T09:08:33.605 INFO:teuthology.orchestra.run.smithi031.stdout:3603 write 0x569bcc8 thru 0x56a0151 (0x448a bytes) 2024-09-13T09:08:33.614 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.614 INFO:teuthology.orchestra.run.smithi031.stdout:3608 punch from 0x3bb80e1 to 0x3bbfb28, (0x7a47 bytes) 2024-09-13T09:08:33.627 INFO:teuthology.orchestra.run.smithi031.stdout:3609 trunc from 0xebad8bf to 0x5c1e5c2 2024-09-13T09:08:33.657 INFO:teuthology.orchestra.run.smithi031.stdout:3610 write 0x8a47ee7 thru 0x8a4cf8d (0x50a7 bytes) 2024-09-13T09:08:33.668 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:33.663+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:33.669 INFO:teuthology.orchestra.run.smithi031.stdout:3612 punch from 0x1dd20de to 0x1de123f, (0xf161 bytes) 2024-09-13T09:08:33.669 INFO:teuthology.orchestra.run.smithi031.stdout:3613 punch from 0xc6cc50 to 0xc7cb1c, (0xfecc bytes) 2024-09-13T09:08:33.669 INFO:teuthology.orchestra.run.smithi031.stdout:3614 read 0x60c6e1 thru 0x61389a (0x71ba bytes) 2024-09-13T09:08:33.681 INFO:teuthology.orchestra.run.smithi031.stdout:3615 write 0xa9cf523 thru 0xa9da6e6 (0xb1c4 bytes) 2024-09-13T09:08:33.704 INFO:teuthology.orchestra.run.smithi031.stdout:3617 trunc from 0xa9da6e7 to 0xe5b97cc 2024-09-13T09:08:33.714 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:33.715 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:33.715 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:33.715 INFO:teuthology.orchestra.run.smithi031.stdout:3621 writesame 0x509b20b thru 0x50a5d76 data_size 0x17(0xab6c bytes) 2024-09-13T09:08:33.727 INFO:teuthology.orchestra.run.smithi031.stdout:3622 write 0x4e8d951 thru 0x4e94ea1 (0x7551 bytes) 2024-09-13T09:08:33.733 INFO:teuthology.orchestra.run.smithi031.stdout:3626 writesame 0x510b9be thru 0x51181a4 data_size 0x19(0xc7e7 bytes) 2024-09-13T09:08:33.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:33.739 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:33.739 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:33.744 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.744 INFO:teuthology.orchestra.run.smithi031.stdout:3629 write 0x82a2311 thru 0x82a2a38 (0x728 bytes) 2024-09-13T09:08:33.751 INFO:teuthology.orchestra.run.smithi031.stdout:3630 trunc from 0xe5b97cc to 0xc757b49 2024-09-13T09:08:33.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:33.757 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:33.761 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.761 INFO:teuthology.orchestra.run.smithi031.stdout:3633 write 0x62a739a thru 0x62b44d8 (0xd13f bytes) 2024-09-13T09:08:33.768 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:33.767+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:33.779 INFO:teuthology.orchestra.run.smithi031.stdout:3635 writesame 0x6d1afcf thru 0x6d27ba3 data_size 0x1(0xcbd5 bytes) 2024-09-13T09:08:33.780 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:33.780 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:33.803 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:33.804 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:33.818 INFO:teuthology.orchestra.run.smithi031.stdout:3636 read 0x4d0efec thru 0x4d1981e (0xa833 bytes) 2024-09-13T09:08:33.820 INFO:teuthology.orchestra.run.smithi031.stdout:3637 read 0x9d431c4 thru 0x9d4e01f (0xae5c bytes) 2024-09-13T09:08:33.822 INFO:teuthology.orchestra.run.smithi031.stdout:3638 trunc from 0xc757b49 to 0xdd8fae7 2024-09-13T09:08:33.830 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:33.831 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:33.831 INFO:teuthology.orchestra.run.smithi031.stdout:3640 read 0x52393df thru 0x523b463 (0x2085 bytes) 2024-09-13T09:08:33.833 INFO:teuthology.orchestra.run.smithi031.stdout:3641 write 0x732b271 thru 0x7337c9e (0xca2e bytes) 2024-09-13T09:08:33.852 INFO:teuthology.orchestra.run.smithi031.stdout:3642 write 0x4d002fc thru 0x4d0d41c (0xd121 bytes) 2024-09-13T09:08:33.862 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:33.862 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:33.863 INFO:teuthology.orchestra.run.smithi031.stdout:3643 read 0x2a9bf0b thru 0x2a9d939 (0x1a2f bytes) 2024-09-13T09:08:33.864 INFO:teuthology.orchestra.run.smithi031.stdout:3645 writesame 0x88d50e4 thru 0x88d5c7e data_size 0x1(0xb9b bytes) 2024-09-13T09:08:33.869 INFO:teuthology.orchestra.run.smithi031.stdout:3647 read 0xa2f7de7 thru 0xa2fdf9a (0x61b4 bytes) 2024-09-13T09:08:33.869 INFO:teuthology.orchestra.run.smithi031.stdout:3648 read 0xc3e3d22 thru 0xc3e7600 (0x38df bytes) 2024-09-13T09:08:33.869 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:33.867+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:33.869 INFO:teuthology.orchestra.run.smithi031.stdout:3649 writesame 0x3ac2ec7 thru 0x3ac659b data_size 0x1(0x36d5 bytes) 2024-09-13T09:08:33.884 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:33.884 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:33.885 INFO:teuthology.orchestra.run.smithi031.stdout:3650 read 0x4cf783f thru 0x4cf78c7 (0x89 bytes) 2024-09-13T09:08:33.886 INFO:teuthology.orchestra.run.smithi031.stdout:3651 writesame 0x5052ca7 thru 0x505a7b8 data_size 0x3b(0x7b12 bytes) 2024-09-13T09:08:33.892 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:33.892 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:33.894 INFO:teuthology.orchestra.run.smithi031.stdout:3652 punch from 0xd991f36 to 0xd998edc, (0x6fa6 bytes) 2024-09-13T09:08:33.894 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.894 INFO:teuthology.orchestra.run.smithi031.stdout:3654 read 0x694223 thru 0x6a31e0 (0xefbe bytes) 2024-09-13T09:08:33.895 INFO:teuthology.orchestra.run.smithi031.stdout:3655 write 0x6fb960b thru 0x6fc316d (0x9b63 bytes) 2024-09-13T09:08:33.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:33.902 INFO:teuthology.orchestra.run.smithi031.stdout:3656 write 0xd1670e9 thru 0xd173344 (0xc25c bytes) 2024-09-13T09:08:33.922 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:33.923 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:33.923 INFO:teuthology.orchestra.run.smithi031.stdout:3657 trunc from 0xdd8fae7 to 0x93fe84f 2024-09-13T09:08:33.950 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:33.950 INFO:teuthology.orchestra.run.smithi031.stdout:3660 read 0x581b827 thru 0x582100f (0x57e9 bytes) 2024-09-13T09:08:33.951 INFO:teuthology.orchestra.run.smithi031.stdout:3661 read 0x1d9e4fa thru 0x1dabd6e (0xd875 bytes) 2024-09-13T09:08:33.951 INFO:teuthology.orchestra.run.smithi031.stdout:3662 read 0x47cd882 thru 0x47d0d14 (0x3493 bytes) 2024-09-13T09:08:33.952 INFO:teuthology.orchestra.run.smithi031.stdout:3665 writesame 0x242789a thru 0x242adef data_size 0x1(0x3556 bytes) 2024-09-13T09:08:33.970 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:33.965+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:33.985 INFO:teuthology.orchestra.run.smithi031.stdout:3666 write 0xe12a0c6 thru 0xe12b83e (0x1779 bytes) 2024-09-13T09:08:33.993 INFO:teuthology.orchestra.run.smithi031.stdout:3667 write 0x7f1f372 thru 0x7f2c00d (0xcc9c bytes) 2024-09-13T09:08:34.010 INFO:teuthology.orchestra.run.smithi031.stdout:3669 punch from 0x2b8ba7 to 0x2ba3a5, (0x17fe bytes) 2024-09-13T09:08:34.013 INFO:teuthology.orchestra.run.smithi031.stdout:3670 trunc from 0xe12b83f to 0xbaf9ce4 2024-09-13T09:08:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:3671 punch from 0x5ae3819 to 0x5aea2ef, (0x6ad6 bytes) 2024-09-13T09:08:34.034 INFO:teuthology.orchestra.run.smithi031.stdout:3672 write 0x430e1e5 thru 0x4314b36 (0x6952 bytes) 2024-09-13T09:08:34.037 INFO:teuthology.orchestra.run.smithi031.stdout:3673 writesame 0x3f8c09a thru 0x3f8ec37 data_size 0x3(0x2b9e bytes) 2024-09-13T09:08:34.042 INFO:teuthology.orchestra.run.smithi031.stdout:3678 write 0x738670b thru 0x738670e (0x4 bytes) 2024-09-13T09:08:34.043 INFO:teuthology.orchestra.run.smithi031.stdout:3680 writesame 0x70a519b thru 0x70b4a57 data_size 0x1(0xf8bd bytes) 2024-09-13T09:08:34.063 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:34.063 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:34.071 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:34.071+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:34.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:34.123 INFO:teuthology.orchestra.run.smithi031.stdout:3683 trunc from 0xbaf9ce4 to 0xa772f64 2024-09-13T09:08:34.145 INFO:teuthology.orchestra.run.smithi031.stdout:3684 trunc from 0xa772f64 to 0x712cbf6 2024-09-13T09:08:34.168 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 0.247782 (3f6e) 2024-09-13T09:08:34.169 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:08:34.172 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:34.165+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:34.259 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:34.259 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:34.273 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:34.269+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:34.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:34.374 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:34.371+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:34.439 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:34.440 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:34.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:34.495 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:34.495+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:34.575 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:34.569+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:34.615 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:34.615 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:34.630 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:34.676 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:34.669+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:34.756 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:34.757 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:34.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:34.815 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:34.815+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:34.878 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:34.875+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:34.892 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:34.892 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:34.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:34.991 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:34.991+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:35.065 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:35.065 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:35.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:35.080 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:35.079+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:35.182 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:35.177+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:35.246 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:35.247 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:35.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:35.307 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:35.307+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:35.382 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:35.382 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:35.383 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:35.379+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:35.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:35.487 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:35.487+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:35.563 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:35.563 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:35.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:35.625 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:35.621+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:35.685 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:35.681+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:35.712 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:35.712 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:35.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:35.813 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:35.809+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:35.887 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:35.887+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:35.897 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:35.897 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:35.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:35.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:35.997 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:35.993+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:36.082 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:36.083 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:36.088 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:36.087+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:36.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:36.108 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:36.108 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:36.124 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:36.124 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:36.141 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:36.141 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:36.157 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:36.183 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:36.183 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:36.189 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:36.185+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:36.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:08:36.221 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:36.221 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:36.224 INFO:teuthology.orchestra.run.smithi031.stdout:3685 write 0x15e537a thru 0x15f3c20 (0xe8a7 bytes) 2024-09-13T09:08:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:3686 trunc from 0x712cbf6 to 0x49635be 2024-09-13T09:08:36.252 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:36.253 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:36.275 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.275 INFO:teuthology.orchestra.run.smithi031.stdout:3688 writesame 0x1001a05 thru 0x10062d0 data_size 0x6(0x48cc bytes) 2024-09-13T09:08:36.282 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:36.282 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:36.289 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:36.285+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:36.298 INFO:teuthology.orchestra.run.smithi031.stdout:3689 writesame 0xa37e910 thru 0xa388361 data_size 0x1(0x9a52 bytes) 2024-09-13T09:08:36.298 INFO:teuthology.orchestra.run.smithi031.stdout:3690 read 0x20c0511 thru 0x20cbaf4 (0xb5e4 bytes) 2024-09-13T09:08:36.299 INFO:teuthology.orchestra.run.smithi031.stdout:3691 write 0x5fa2951 thru 0x5fa761d (0x4ccd bytes) 2024-09-13T09:08:36.302 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:36.302 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:36.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:36.325 INFO:teuthology.orchestra.run.smithi031.stdout:3693 trunc from 0xa388362 to 0x4f5804a 2024-09-13T09:08:36.345 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.345 INFO:teuthology.orchestra.run.smithi031.stdout:3695 punch from 0x28dbfba to 0x28e885a, (0xc8a0 bytes) 2024-09-13T09:08:36.366 INFO:teuthology.orchestra.run.smithi031.stdout:3696 write 0xc133c4a thru 0xc139fe4 (0x639b bytes) 2024-09-13T09:08:36.391 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:36.387+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:36.395 INFO:teuthology.orchestra.run.smithi031.stdout:3697 writesame 0xc870603 thru 0xc87a2f0 data_size 0x35(0x9cee bytes) 2024-09-13T09:08:36.395 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.395 INFO:teuthology.orchestra.run.smithi031.stdout:3700 read 0x961e42c thru 0x962b7ee (0xd3c3 bytes) 2024-09-13T09:08:36.396 INFO:teuthology.orchestra.run.smithi031.stdout:3702 punch from 0x89d2a53 to 0x89d7b74, (0x5121 bytes) 2024-09-13T09:08:36.396 INFO:teuthology.orchestra.run.smithi031.stdout:3704 punch from 0x219f8ce to 0x21a79f7, (0x8129 bytes) 2024-09-13T09:08:36.396 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.396 INFO:teuthology.orchestra.run.smithi031.stdout:3706 punch from 0xcf266 to 0xdda6e, (0xe808 bytes) 2024-09-13T09:08:36.396 INFO:teuthology.orchestra.run.smithi031.stdout:3709 write 0x9b689a5 thru 0x9b6be87 (0x34e3 bytes) 2024-09-13T09:08:36.405 INFO:teuthology.orchestra.run.smithi031.stdout:3711 read 0x4b359ef thru 0x4b38534 (0x2b46 bytes) 2024-09-13T09:08:36.416 INFO:teuthology.orchestra.run.smithi031.stdout:3713 write 0xdbf0a47 thru 0xdbf1d47 (0x1301 bytes) 2024-09-13T09:08:36.423 INFO:teuthology.orchestra.run.smithi031.stdout:3714 read 0x2716f7b thru 0x272050b (0x9591 bytes) 2024-09-13T09:08:36.424 INFO:teuthology.orchestra.run.smithi031.stdout:3715 writesame 0x9b3b292 thru 0x9b492f4 data_size 0x1f(0xe063 bytes) 2024-09-13T09:08:36.450 INFO:teuthology.orchestra.run.smithi031.stdout:3716 write 0xe4826d2 thru 0xe491c26 (0xf555 bytes) 2024-09-13T09:08:36.471 INFO:teuthology.orchestra.run.smithi031.stdout:3718 trunc from 0xe491c27 to 0x99445f3 2024-09-13T09:08:36.490 INFO:teuthology.orchestra.run.smithi031.stdout:3719 writesame 0x35ef45e thru 0x35f35e0 data_size 0x1(0x4183 bytes) 2024-09-13T09:08:36.491 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:36.485+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:36.507 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:36.507 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:36.507 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:36.507 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:36.507 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:36.515 INFO:teuthology.orchestra.run.smithi031.stdout:3720 trunc from 0x99445f3 to 0xab28af 2024-09-13T09:08:36.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:36.541 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.549 INFO:teuthology.orchestra.run.smithi031.stdout:3723 trunc from 0xab28af to 0x690f810 2024-09-13T09:08:36.555 INFO:teuthology.orchestra.run.smithi031.stdout:3724 read 0x17aa330 thru 0x17abbc9 (0x189a bytes) 2024-09-13T09:08:36.556 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.557 INFO:teuthology.orchestra.run.smithi031.stdout:3727 trunc from 0x690f810 to 0x7c02f1c 2024-09-13T09:08:36.563 INFO:teuthology.orchestra.run.smithi031.stdout:3730 punch from 0x3b5338e to 0x3b5cee6, (0x9b58 bytes) 2024-09-13T09:08:36.567 INFO:teuthology.orchestra.run.smithi031.stdout:3731 trunc from 0x7c02f1c to 0x46ccb53 2024-09-13T09:08:36.584 INFO:teuthology.orchestra.run.smithi031.stdout:3732 write 0x72b0002 thru 0x72b7f10 (0x7f0f bytes) 2024-09-13T09:08:36.598 INFO:teuthology.orchestra.run.smithi031.stdout:3733 punch from 0x38e6db8 to 0x38e7154, (0x39c bytes) 2024-09-13T09:08:36.612 INFO:teuthology.orchestra.run.smithi031.stdout:3734 writesame 0xdd8fba9 thru 0xdd904b5 data_size 0x1(0x90d bytes) 2024-09-13T09:08:36.617 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:36.613+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:36.628 INFO:teuthology.orchestra.run.smithi031.stdout:3736 trunc from 0xdd904b6 to 0x1704011 2024-09-13T09:08:36.664 INFO:teuthology.orchestra.run.smithi031.stdout:3737 writesame 0xde0b46a thru 0xde168d0 data_size 0x1(0xb467 bytes) 2024-09-13T09:08:36.664 INFO:teuthology.orchestra.run.smithi031.stdout:3738 punch from 0x4b35ec9 to 0x4b455cd, (0xf704 bytes) 2024-09-13T09:08:36.664 INFO:teuthology.orchestra.run.smithi031.stdout:3739 write 0x9cd09c4 thru 0x9cdc219 (0xb856 bytes) 2024-09-13T09:08:36.683 INFO:teuthology.orchestra.run.smithi031.stdout:3740 punch from 0xd22ce29 to 0xd23504d, (0x8224 bytes) 2024-09-13T09:08:36.683 INFO:teuthology.orchestra.run.smithi031.stdout:3741 punch from 0x158ddee to 0x158e0e1, (0x2f3 bytes) 2024-09-13T09:08:36.683 INFO:teuthology.orchestra.run.smithi031.stdout:3742 write 0x6953392 thru 0x695a712 (0x7381 bytes) 2024-09-13T09:08:36.692 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:36.685+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:36.699 INFO:teuthology.orchestra.run.smithi031.stdout:3744 punch from 0x5e088c9 to 0x5e16e98, (0xe5cf bytes) 2024-09-13T09:08:36.699 INFO:teuthology.orchestra.run.smithi031.stdout:3745 read 0x96e8093 thru 0x96f15c2 (0x9530 bytes) 2024-09-13T09:08:36.700 INFO:teuthology.orchestra.run.smithi031.stdout:3746 write 0x63a720d thru 0x63b668a (0xf47e bytes) 2024-09-13T09:08:36.708 INFO:teuthology.orchestra.run.smithi031.stdout:3748 punch from 0x3c7ed7d to 0x3c8c15f, (0xd3e2 bytes) 2024-09-13T09:08:36.708 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.708 INFO:teuthology.orchestra.run.smithi031.stdout:3751 writesame 0x95bce67 thru 0x95cad10 data_size 0xb(0xdeaa bytes) 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.087768316999998, 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.087744987000001, 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.714 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:08:36.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.087721969999997, 2024-09-13T09:08:36.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:36.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:08:36.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.087698840000002, 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:08:36.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.727 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.085082077000003, 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.728 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:08:36.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.732 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.085057990999999, 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:08:36.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.085031338, 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.737 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:08:36.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:08:36.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:08:36.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:08:36.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:36.744 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.085007697999998, 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:36.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:08:36.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:08:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:08:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.749 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.081192156999997, 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.754 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.081168613999999, 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:08:36.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.081145323999998, 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:36.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.763 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.081121811000003, 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:36.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.081063354999998, 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.772 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.077032418999998, 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 43.757134121999997, 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:08:36.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:08:36.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.782 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 43.754762343000003, 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:36.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:08:36.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:08:36.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 43.754228427999998, 2024-09-13T09:08:36.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:08:36.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 43.750827905000001, 2024-09-13T09:08:36.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:36.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:08:36.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 43.749828098000002, 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:08:36.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:08:36.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.802 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 43.746657820000003, 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:36.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:08:36.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:36.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.807 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.808 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.808 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:36.808 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:36.808 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:36.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:3752 read 0x769c986 thru 0x76abc7d (0xf2f8 bytes) 2024-09-13T09:08:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:3753 write 0x7d67e8c thru 0x7d69402 (0x1577 bytes) 2024-09-13T09:08:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:3754 punch from 0x1a3772f to 0x1a377c9, (0x9a bytes) 2024-09-13T09:08:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:3756 writesame 0x5cdba9b thru 0x5cdc118 data_size 0x3(0x67e bytes) 2024-09-13T09:08:36.811 INFO:teuthology.orchestra.run.smithi031.stdout:3757 trunc from 0xde168d1 to 0xb249abb 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:3758 read 0x2df0c4e thru 0x2df7d81 (0x7134 bytes) 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:3759 read 0x6d3dea5 thru 0x6d493a2 (0xb4fe bytes) 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:3763 writesame 0x6ba5ac1 thru 0x6baa4b4 data_size 0x2(0x49f4 bytes) 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:3764 punch from 0x2acdcdd to 0x2ad318c, (0x54af bytes) 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:3771 trunc from 0xb249abb to 0x2bfb496 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:3773 read 0x2163b88 thru 0x2170628 (0xcaa1 bytes) 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:3775 punch from 0x11a0199 to 0x11a5a85, (0x58ec bytes) 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.812 INFO:teuthology.orchestra.run.smithi031.stdout:3777 write 0x9e29458 thru 0x9e340fa (0xaca3 bytes) 2024-09-13T09:08:36.813 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:36.807+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:36.823 INFO:teuthology.orchestra.run.smithi031.stdout:3779 read 0x820abe7 thru 0x821a650 (0xfa6a bytes) 2024-09-13T09:08:36.823 INFO:teuthology.orchestra.run.smithi031.stdout:3782 punch from 0x599d39f to 0x59a1a71, (0x46d2 bytes) 2024-09-13T09:08:36.823 INFO:teuthology.orchestra.run.smithi031.stdout:3783 write 0x4a50e04 thru 0x4a5e60c (0xd809 bytes) 2024-09-13T09:08:36.858 INFO:teuthology.orchestra.run.smithi031.stdout:3784 writesame 0xea6e283 thru 0xea75fd5 data_size 0x1(0x7d53 bytes) 2024-09-13T09:08:36.858 INFO:teuthology.orchestra.run.smithi031.stdout:3785 writesame 0x5dc62fd thru 0x5dca6d0 data_size 0x6(0x43d4 bytes) 2024-09-13T09:08:36.869 INFO:teuthology.orchestra.run.smithi031.stdout:3787 write 0x895fbd5 thru 0x896a9e2 (0xae0e bytes) 2024-09-13T09:08:36.891 INFO:teuthology.orchestra.run.smithi031.stdout:3788 read 0xdcb4921 thru 0xdcbd778 (0x8e58 bytes) 2024-09-13T09:08:36.891 INFO:teuthology.orchestra.run.smithi031.stdout:3790 read 0xc966a1c thru 0xc96e4bf (0x7aa4 bytes) 2024-09-13T09:08:36.891 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:36.909 INFO:teuthology.orchestra.run.smithi031.stdout:3793 trunc from 0xea75fd6 to 0xde53aa 2024-09-13T09:08:36.910 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:36.905+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:36.960 INFO:teuthology.orchestra.run.smithi031.stdout:3794 writesame 0xe75b047 thru 0xe7665be data_size 0x4(0xb578 bytes) 2024-09-13T09:08:36.960 INFO:teuthology.orchestra.run.smithi031.stdout:3795 write 0xe4ec8d6 thru 0xe4f7183 (0xa8ae bytes) 2024-09-13T09:08:36.979 INFO:teuthology.orchestra.run.smithi031.stdout:3797 read 0x4f91a76 thru 0x4f9ad94 (0x931f bytes) 2024-09-13T09:08:36.979 INFO:teuthology.orchestra.run.smithi031.stdout:3798 read 0x3c1931e thru 0x3c1a8d8 (0x15bb bytes) 2024-09-13T09:08:36.979 INFO:teuthology.orchestra.run.smithi031.stdout:3799 read 0x3ce5e37 thru 0x3cf1b07 (0xbcd1 bytes) 2024-09-13T09:08:36.979 INFO:teuthology.orchestra.run.smithi031.stdout:3802 read 0xdbd7733 thru 0xdbe6e45 (0xf713 bytes) 2024-09-13T09:08:36.979 INFO:teuthology.orchestra.run.smithi031.stdout:3804 writesame 0xa3cfa4c thru 0xa3d2bca data_size 0x1(0x317f bytes) 2024-09-13T09:08:36.993 INFO:teuthology.orchestra.run.smithi031.stdout:3806 writesame 0x5e8e769 thru 0x5e95d9b data_size 0x1(0x7633 bytes) 2024-09-13T09:08:36.995 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:36.995 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:36.996 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:37.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:37.012 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:37.011+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:37.029 INFO:teuthology.orchestra.run.smithi031.stdout:3807 writesame 0x3369ac5 thru 0x3371396 data_size 0xf(0x78d2 bytes) 2024-09-13T09:08:37.047 INFO:teuthology.orchestra.run.smithi031.stdout:3810 read 0x8026cd thru 0x805647 (0x2f7b bytes) 2024-09-13T09:08:37.047 INFO:teuthology.orchestra.run.smithi031.stdout:3811 write 0x794ea0d thru 0x795b438 (0xca2c bytes) 2024-09-13T09:08:37.071 INFO:teuthology.orchestra.run.smithi031.stdout:3815 write 0x59ec1e thru 0x5ad2c3 (0xe6a6 bytes) 2024-09-13T09:08:37.076 INFO:teuthology.orchestra.run.smithi031.stdout:3816 read 0xca414e5 thru 0xca49b52 (0x866e bytes) 2024-09-13T09:08:37.076 INFO:teuthology.orchestra.run.smithi031.stdout:3817 writesame 0xab6f3da thru 0xab70841 data_size 0x4(0x1468 bytes) 2024-09-13T09:08:37.083 INFO:teuthology.orchestra.run.smithi031.stdout:3818 write 0x1081f98 thru 0x1083f59 (0x1fc2 bytes) 2024-09-13T09:08:37.092 INFO:teuthology.orchestra.run.smithi031.stdout:3819 write 0x8dc237d thru 0x8dc6098 (0x3d1c bytes) 2024-09-13T09:08:37.111 INFO:teuthology.orchestra.run.smithi031.stdout:3820 trunc from 0xe7665bf to 0xaf59a24 2024-09-13T09:08:37.111 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:37.105+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:37.151 INFO:teuthology.orchestra.run.smithi031.stdout:3821 write 0xd72a919 thru 0xd72d4f1 (0x2bd9 bytes) 2024-09-13T09:08:37.162 INFO:teuthology.orchestra.run.smithi031.stdout:3822 read 0xcf48bc3 thru 0xcf4d5ae (0x49ec bytes) 2024-09-13T09:08:37.163 INFO:teuthology.orchestra.run.smithi031.stdout:3823 writesame 0x1e37af9 thru 0x1e4735b data_size 0x1(0xf863 bytes) 2024-09-13T09:08:37.188 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:37.188 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:37.189 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:37.189 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:37.189 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:37.199 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:08:37.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:37.212 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:37.211+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:37.313 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:37.311+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:37.370 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927801959, 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.371 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:37.375 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927777376000002, 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:37.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:37.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927753371999998, 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:08:37.380 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:08:37.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.384 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927729888999998, 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:08:37.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927706481999998, 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:37.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927683233000003, 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:08:37.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927651126999997, 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.398 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:08:37.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:08:37.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.402 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927615510000003, 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:37.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.92758353, 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.407 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:08:37.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:08:37.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927514250999998, 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.412 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:37.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:08:37.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927489190999999, 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.416 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:08:37.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927465761000001, 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:08:37.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927442378000002, 2024-09-13T09:08:37.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:37.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:08:37.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.429 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927418961999997, 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.430 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:08:37.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927395038999997, 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:08:37.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:08:37.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.439 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927361955000002, 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:08:37.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927338155000001, 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:08:37.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:08:37.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:08:37.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927313828999999, 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:08:37.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:08:37.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927281268999998, 2024-09-13T09:08:37.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:08:37.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:08:37.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.457 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 61.927241395999999, 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.458 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.462 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:37.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:37.465 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:37.457+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:37.565 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:37.563+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:37.641 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:37.641 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:37.641 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:37.641 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:37.655 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:37.701 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:37.697+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:37.766 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:37.763+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:37.772 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:37.772 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:37.772 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:37.772 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:37.773 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:37.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:37.867 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:37.867+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:37.950 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724331793000005, 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:08:37.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.955 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724300943000003, 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.956 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:08:37.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.960 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724274776000001, 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724235301999997, 2024-09-13T09:08:37.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:08:37.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724200748000001, 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.970 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:08:37.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:08:37.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.975 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724168391999996, 2024-09-13T09:08:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:08:37.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724136121000001, 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:08:37.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724102084999998, 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:08:37.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:08:37.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724068821000003, 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.994 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.724036581000007, 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.995 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:37.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:08:37.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:37.999 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.715056494999999, 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:38.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.715015614999999, 2024-09-13T09:08:38.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:38.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:08:38.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.714975347000006, 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.012 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.714928303999997, 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.013 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:08:38.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:08:38.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.714882742, 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.714792574000001, 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:08:38.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.026 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.714746020000007, 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:38.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:08:38.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.713105096000007, 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:08:38.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:08:38.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:08:38.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.713039289999998, 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:38.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:08:38.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:08:38.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 5.0300316450000002, 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:38.040 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:38.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:38.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:38.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:08:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:08:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:38.046 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:38.046 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:38.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:38.048 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:38.047+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:38.148 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:38.141+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:38.183 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:38.184 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:38.184 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:38.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:38.249 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:38.245+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:38.253 INFO:teuthology.orchestra.run.smithi031.stdout:3824 read 0xb7ca40d thru 0xb7d1e68 (0x7a5c bytes) 2024-09-13T09:08:38.253 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.253 INFO:teuthology.orchestra.run.smithi031.stdout:3829 read 0x946198c thru 0x946bf5d (0xa5d2 bytes) 2024-09-13T09:08:38.253 INFO:teuthology.orchestra.run.smithi031.stdout:3830 punch from 0x58a7c4d to 0x58aaf01, (0x32b4 bytes) 2024-09-13T09:08:38.253 INFO:teuthology.orchestra.run.smithi031.stdout:3831 read 0x7ac6972 thru 0x7ad0a71 (0xa100 bytes) 2024-09-13T09:08:38.266 INFO:teuthology.orchestra.run.smithi031.stdout:3832 trunc from 0xd72d4f2 to 0x72f1221 2024-09-13T09:08:38.309 INFO:teuthology.orchestra.run.smithi031.stdout:3833 write 0x2f3a92b thru 0x2f47423 (0xcaf9 bytes) 2024-09-13T09:08:38.317 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:38.320 INFO:teuthology.orchestra.run.smithi031.stdout:3834 punch from 0x4667049 to 0x46708ad, (0x9864 bytes) 2024-09-13T09:08:38.320 INFO:teuthology.orchestra.run.smithi031.stdout:3835 read 0x613a23d thru 0x614246b (0x822f bytes) 2024-09-13T09:08:38.336 INFO:teuthology.orchestra.run.smithi031.stdout:3837 write 0xa4b65b5 thru 0xa4bd5cc (0x7018 bytes) 2024-09-13T09:08:38.388 INFO:teuthology.orchestra.run.smithi031.stdout:3838 read 0x5d50758 thru 0x5d5b536 (0xaddf bytes) 2024-09-13T09:08:38.389 INFO:teuthology.orchestra.run.smithi031.stdout:3839 punch from 0x7ebeee5 to 0x7ec6cbf, (0x7dda bytes) 2024-09-13T09:08:38.389 INFO:teuthology.orchestra.run.smithi031.stdout:3840 write 0x6b11619 thru 0x6b19029 (0x7a11 bytes) 2024-09-13T09:08:38.389 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:38.389 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:38.390 INFO:teuthology.orchestra.run.smithi031.stdout:3841 write 0x15487f9 thru 0x15551fc (0xca04 bytes) 2024-09-13T09:08:38.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:38.391 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:38.381+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:38.400 INFO:teuthology.orchestra.run.smithi031.stdout:3842 writesame 0xd8f4387 thru 0xd8fe2d7 data_size 0x5(0x9f51 bytes) 2024-09-13T09:08:38.400 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.401 INFO:teuthology.orchestra.run.smithi031.stdout:3844 writesame 0xc11969d thru 0xc12677b data_size 0x1(0xd0df bytes) 2024-09-13T09:08:38.439 INFO:teuthology.orchestra.run.smithi031.stdout:3845 write 0x257c7dd thru 0x25849f6 (0x821a bytes) 2024-09-13T09:08:38.457 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.457 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.457 INFO:teuthology.orchestra.run.smithi031.stdout:3848 read 0x6246b47 thru 0x62516a0 (0xab5a bytes) 2024-09-13T09:08:38.470 INFO:teuthology.orchestra.run.smithi031.stdout:3853 trunc from 0xd8fe2d8 to 0x68f57ac 2024-09-13T09:08:38.488 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:38.481+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:38.494 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:38.494 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:38.502 INFO:teuthology.orchestra.run.smithi031.stdout:3854 read 0x402e8c thru 0x40dd80 (0xaef5 bytes) 2024-09-13T09:08:38.503 INFO:teuthology.orchestra.run.smithi031.stdout:3855 write 0x5dfd2b3 thru 0x5dfed91 (0x1adf bytes) 2024-09-13T09:08:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.510 INFO:teuthology.orchestra.run.smithi031.stdout:3858 writesame 0x50cd1c7 thru 0x50d3882 data_size 0x32(0x66bc bytes) 2024-09-13T09:08:38.514 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:38.515 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:38.537 INFO:teuthology.orchestra.run.smithi031.stdout:3860 writesame 0x8fcd7de thru 0x8fd8925 data_size 0xb7(0xb148 bytes) 2024-09-13T09:08:38.537 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.537 INFO:teuthology.orchestra.run.smithi031.stdout:3864 read 0x30dac2d thru 0x30df6f9 (0x4acd bytes) 2024-09-13T09:08:38.538 INFO:teuthology.orchestra.run.smithi031.stdout:3866 read 0x726f80a thru 0x72725f1 (0x2de8 bytes) 2024-09-13T09:08:38.538 INFO:teuthology.orchestra.run.smithi031.stdout:3867 read 0x5632c85 thru 0x5633883 (0xbff bytes) 2024-09-13T09:08:38.547 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:38.547 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:38.551 INFO:teuthology.orchestra.run.smithi031.stdout:3868 write 0xb40c5e2 thru 0xb414d63 (0x8782 bytes) 2024-09-13T09:08:38.562 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:38.562 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:38.565 INFO:teuthology.orchestra.run.smithi031.stdout:3871 read 0x57de39d thru 0x57ebbb2 (0xd816 bytes) 2024-09-13T09:08:38.567 INFO:teuthology.orchestra.run.smithi031.stdout:3872 writesame 0x867acdf thru 0x8684e13 data_size 0x1(0xa135 bytes) 2024-09-13T09:08:38.572 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:38.572 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:38.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:38.601 INFO:teuthology.orchestra.run.smithi031.stdout:3875 punch from 0x3fabffd to 0x3fb90e7, (0xd0ea bytes) 2024-09-13T09:08:38.601 INFO:teuthology.orchestra.run.smithi031.stdout:3876 write 0xb2f54a9 thru 0xb2facb0 (0x5808 bytes) 2024-09-13T09:08:38.601 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:38.599+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:38.603 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:38.603 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:38.607 INFO:teuthology.orchestra.run.smithi031.stdout:3877 write 0x4acd72d thru 0x4adad6d (0xd641 bytes) 2024-09-13T09:08:38.627 INFO:teuthology.orchestra.run.smithi031.stdout:3878 write 0x7e2e7a1 thru 0x7e3b266 (0xcac6 bytes) 2024-09-13T09:08:38.631 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:38.631 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:38.658 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:38.658 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:38.659 INFO:teuthology.orchestra.run.smithi031.stdout:3879 write 0xca581de thru 0xca66fdf (0xee02 bytes) 2024-09-13T09:08:38.678 INFO:teuthology.orchestra.run.smithi031.stdout:3880 punch from 0x424714e to 0x424a606, (0x34b8 bytes) 2024-09-13T09:08:38.678 INFO:teuthology.orchestra.run.smithi031.stdout:3882 read 0x4293b73 thru 0x429c027 (0x84b5 bytes) 2024-09-13T09:08:38.679 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.679 INFO:teuthology.orchestra.run.smithi031.stdout:3885 write 0x8dd6990 thru 0x8ddcd7a (0x63eb bytes) 2024-09-13T09:08:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:3886 writesame 0x7644ae4 thru 0x764c878 data_size 0x1(0x7d95 bytes) 2024-09-13T09:08:38.687 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:38.687+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:38.693 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:38.694 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:38.710 INFO:teuthology.orchestra.run.smithi031.stdout:3887 punch from 0x40cef3c to 0x40d2c58, (0x3d1c bytes) 2024-09-13T09:08:38.710 INFO:teuthology.orchestra.run.smithi031.stdout:3888 read 0x42bc473 thru 0x42cc098 (0xfc26 bytes) 2024-09-13T09:08:38.711 INFO:teuthology.orchestra.run.smithi031.stdout:3889 read 0x9f20514 thru 0x9f254f0 (0x4fdd bytes) 2024-09-13T09:08:38.711 INFO:teuthology.orchestra.run.smithi031.stdout:3890 write 0x2a5ca1e thru 0x2a5f3ec (0x29cf bytes) 2024-09-13T09:08:38.714 INFO:teuthology.orchestra.run.smithi031.stdout:3891 punch from 0x3abd6c5 to 0x3aca1e6, (0xcb21 bytes) 2024-09-13T09:08:38.714 INFO:teuthology.orchestra.run.smithi031.stdout:3894 punch from 0x2c73db6 to 0x2c7f6c2, (0xb90c bytes) 2024-09-13T09:08:38.722 INFO:teuthology.orchestra.run.smithi031.stdout:3895 trunc from 0xca66fe0 to 0x791257e 2024-09-13T09:08:38.757 INFO:teuthology.orchestra.run.smithi031.stdout:3896 writesame 0x61c2e3d thru 0x61cb601 data_size 0x1(0x87c5 bytes) 2024-09-13T09:08:38.760 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:38.760 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:38.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:38.786 INFO:teuthology.orchestra.run.smithi031.stdout:3898 punch from 0x1de3088 to 0x1de878f, (0x5707 bytes) 2024-09-13T09:08:38.786 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:3900 punch from 0x1b0a20c to 0x1b11c34, (0x7a28 bytes) 2024-09-13T09:08:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:3902 read 0x63e590 thru 0x63f8f1 (0x1362 bytes) 2024-09-13T09:08:38.787 INFO:teuthology.orchestra.run.smithi031.stdout:3904 read 0x705da7d thru 0x706ad08 (0xd28c bytes) 2024-09-13T09:08:38.788 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:38.781+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:38.812 INFO:teuthology.orchestra.run.smithi031.stdout:3905 write 0xe1f73c0 thru 0xe202d5d (0xb99e bytes) 2024-09-13T09:08:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:3908 read 0x64e7406 thru 0x64f0767 (0x9362 bytes) 2024-09-13T09:08:38.833 INFO:teuthology.orchestra.run.smithi031.stdout:3911 writesame 0xdc2ea07 thru 0xdc38e4f data_size 0x9(0xa449 bytes) 2024-09-13T09:08:38.843 INFO:teuthology.orchestra.run.smithi031.stdout:3912 write 0x425114e thru 0x425cf59 (0xbe0c bytes) 2024-09-13T09:08:38.848 INFO:teuthology.orchestra.run.smithi031.stdout:3914 punch from 0x4f2079c to 0x4f2ff7b, (0xf7df bytes) 2024-09-13T09:08:38.849 INFO:teuthology.orchestra.run.smithi031.stdout:3915 read 0xa8fa31f thru 0xa905eb3 (0xbb95 bytes) 2024-09-13T09:08:38.853 INFO:teuthology.orchestra.run.smithi031.stdout:3916 trunc from 0xe202d5e to 0xb945ab5 2024-09-13T09:08:38.874 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.874 INFO:teuthology.orchestra.run.smithi031.stdout:3920 writesame 0x11bb5e0 thru 0x11bf78f data_size 0x8(0x41b0 bytes) 2024-09-13T09:08:38.879 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:38.879 INFO:teuthology.orchestra.run.smithi031.stdout:3925 writesame 0x759164 thru 0x75b6bb data_size 0x14(0x2558 bytes) 2024-09-13T09:08:38.883 INFO:teuthology.orchestra.run.smithi031.stdout:3928 write 0x52a6c5f thru 0x52b1394 (0xa736 bytes) 2024-09-13T09:08:38.889 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:38.885+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:38.897 INFO:teuthology.orchestra.run.smithi031.stdout:3929 write 0x2d63fee thru 0x2d6de08 (0x9e1b bytes) 2024-09-13T09:08:38.904 INFO:teuthology.orchestra.run.smithi031.stdout:3930 read 0x8e810af thru 0x8e8230d (0x125f bytes) 2024-09-13T09:08:38.922 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:38.923 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:38.928 INFO:teuthology.orchestra.run.smithi031.stdout:3931 trunc from 0xb945ab5 to 0x3dd354 2024-09-13T09:08:38.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:38.989 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:38.987+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:38.990 INFO:teuthology.orchestra.run.smithi031.stdout:3933 read 0x14dd0e thru 0x15a9e0 (0xccd3 bytes) 2024-09-13T09:08:38.992 INFO:teuthology.orchestra.run.smithi031.stdout:3934 clone 30 order 20 su 4096 sc 15 2024-09-13T09:08:39.091 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:39.085+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:39.104 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:39.104 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:39.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:39.191 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:39.185+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:39.196 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:08:39.197 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:08:39.197 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:08:39.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:08:39.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:08:39.295 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:39.295+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:39.298 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:39.298 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:39.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:39.377 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-13T09:08:39.393 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:39.389+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:39.470 INFO:teuthology.orchestra.run.smithi031.stdout:3935 read 0x34fb3d thru 0x352f6a (0x342e bytes) 2024-09-13T09:08:39.478 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:39.478 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:39.493 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:39.491+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:39.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:39.500 INFO:teuthology.orchestra.run.smithi031.stdout:3936 write 0xa332ddb thru 0xa33a737 (0x795d bytes) 2024-09-13T09:08:39.519 INFO:teuthology.orchestra.run.smithi031.stdout:3937 read 0x5df6fa0 thru 0x5e0159f (0xa600 bytes) 2024-09-13T09:08:39.519 INFO:teuthology.orchestra.run.smithi031.stdout:3938 write 0x52f59ac thru 0x52ffbcb (0xa220 bytes) 2024-09-13T09:08:39.537 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:39.537 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:39.537 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 88, 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T09:06:44.149006+0000", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T09:08:39.195139+0000", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T09:06:55.611487+0000", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T09:07:49.811063+0000", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T09:08:39.538 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T09:08:39.539 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:06:55.131330+0000", 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:39.540 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T09:08:39.541 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:39.542 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.543 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T09:08:39.544 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:39.545 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:02.743102+0000", 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:39.546 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:39.547 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:39.548 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:39.549 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:39.550 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 5, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 5, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.75, 2024-09-13T09:08:39.551 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 3.75, 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 3.75, 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:12.517158+0000", 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:08:39.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:39.553 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.554 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "88", 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 33, 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 88, 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:39.555 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:39.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:39.557 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.558 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "33c6fc7d-9828-4500-b103-b96d1cd7f420", 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.247772216796875, 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:39.559 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 49, 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T09:08:39.560 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:39.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6818/3291756747", 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6822/3291756747", 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6830/3291756747", 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6826/3291756747", 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:39.564 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:08:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "869ea3c5-8e6e-4b49-b70e-083f242a7aae", 2024-09-13T09:08:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:39.565 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.999481201171875, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:39.568 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.569 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/533520566", 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/533520566", 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/533520566", 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/533520566", 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e144d496-76ee-46e0-bdc6-135c77128535", 2024-09-13T09:08:39.571 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:39.572 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.574 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:39.575 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6819/2253604482", 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6823/2253604482", 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/2253604482", 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6827/2253604482", 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "696b9414-9c19-4984-a8ff-14548e46b357", 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.579 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:39.580 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/3367214693", 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/3367214693", 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/3367214693", 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/3367214693", 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.584 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ac58040d-9b54-42ca-b143-1816abb54816", 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:39.585 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6800", 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6801", 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:39.586 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6802", 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6803", 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.587 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6806", 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6807", 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6804", 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6805", 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:39.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6801/3778074750", 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6803/3778074750", 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6807/3778074750", 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6805/3778074750", 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2", 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 52, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:39.591 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6808", 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6809", 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6811", 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6813", 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6819", 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6824", 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:39.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6815", 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.595 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6817", 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6809/1926603087", 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6813/1926603087", 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6824/1926603087", 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6817/1926603087", 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:39.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d", 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:39.597 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6820", 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6821", 2024-09-13T09:08:39.598 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6823", 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6825", 2024-09-13T09:08:39.599 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6830", 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6831", 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6827", 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:39.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6828", 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6821/766305500", 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6825/766305500", 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6831/766305500", 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6828/766305500", 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:39.602 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f36c332c-9497-47e8-b943-f6b6dcaebeb8", 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:39.603 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6810", 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:39.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6812", 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:39.605 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6814", 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6816", 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:39.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6826", 2024-09-13T09:08:39.607 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6829", 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:39.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6818", 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6822", 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.609 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6812/1032956039", 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6816/1032956039", 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6829/1032956039", 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6822/1032956039", 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.610 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.183436+0000", 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.264080+0000", 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:39.612 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.253587+0000", 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:08:39.613 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.244429+0000", 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:39.614 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.369458+0000", 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:39.615 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.158089+0000", 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.217295+0000", 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:39.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.332254+0000", 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:39.617 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.a", 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: 4, 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: 2 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T09:08:39.618 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T09:08:39.619 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T09:08:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:08:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:08:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T09:08:39.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:39.620 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:39.621 INFO:teuthology.orchestra.run.smithi031.stdout:3939 punch from 0x4582cc7 to 0x458c954, (0x9c8d bytes) 2024-09-13T09:08:39.621 INFO:teuthology.orchestra.run.smithi031.stdout:3941 writesame 0x285e9af thru 0x28699da data_size 0x2c(0xb02c bytes) 2024-09-13T09:08:39.622 INFO:tasks.thrashosds.thrasher:j is {'epoch': 88, 'fsid': 'fe16bf6d-4166-4417-80fd-1123617a33d3', 'created': '2024-09-13T09:06:44.149006+0000', 'modified': '2024-09-13T09:08:39.195139+0000', 'last_up_change': '2024-09-13T09:06:55.611487+0000', 'last_in_change': '2024-09-13T09:07:49.811063+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:06:55.131330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:07:02.743102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:07:12.517158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 88, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '33c6fc7d-9828-4500-b103-b96d1cd7f420', 'up': 1, 'in': 1, 'weight': 0.247772216796875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6818', 'nonce': 3291756747}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6822', 'nonce': 3291756747}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6830', 'nonce': 3291756747}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6826', 'nonce': 3291756747}]}, 'public_addr': '172.21.15.31:6818/3291756747', 'cluster_addr': '172.21.15.31:6822/3291756747', 'heartbeat_back_addr': '172.21.15.31:6830/3291756747', 'heartbeat_front_addr': '172.21.15.31:6826/3291756747', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.999481201171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 533520566}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 533520566}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 533520566}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 533520566}]}, 'public_addr': '172.21.15.31:6801/533520566', 'cluster_addr': '172.21.15.31:6803/533520566', 'heartbeat_back_addr': '172.21.15.31:6807/533520566', 'heartbeat_front_addr': '172.21.15.31:6805/533520566', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e144d496-76ee-46e0-bdc6-135c77128535', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6817', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2253604482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6821', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2253604482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6829', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 2253604482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6825', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 2253604482}]}, 'public_addr': '172.21.15.31:6819/2253604482', 'cluster_addr': '172.21.15.31:6823/2253604482', 'heartbeat_back_addr': '172.21.15.31:6831/2253604482', 'heartbeat_front_addr': '172.21.15.31:6827/2253604482', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '696b9414-9c19-4984-a8ff-14548e46b357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 3367214693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 3367214693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 3367214693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 3367214693}]}, 'public_addr': '172.21.15.31:6809/3367214693', 'cluster_addr': '172.21.15.31:6811/3367214693', 'heartbeat_back_addr': '172.21.15.31:6815/3367214693', 'heartbeat_front_addr': '172.21.15.31:6813/3367214693', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ac58040d-9b54-42ca-b143-1816abb54816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6800', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6801', 'nonce': 3778074750}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 3778074750}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 3778074750}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 3778074750}]}, 'public_addr': '172.21.15.60:6801/3778074750', 'cluster_addr': '172.21.15.60:6803/3778074750', 'heartbeat_back_addr': '172.21.15.60:6807/3778074750', 'heartbeat_front_addr': '172.21.15.60:6805/3778074750', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1926603087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6811', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 1926603087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6819', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6824', 'nonce': 1926603087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6815', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 1926603087}]}, 'public_addr': '172.21.15.60:6809/1926603087', 'cluster_addr': '172.21.15.60:6813/1926603087', 'heartbeat_back_addr': '172.21.15.60:6824/1926603087', 'heartbeat_front_addr': '172.21.15.60:6817/1926603087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 766305500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6823', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 766305500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 766305500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6827', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6828', 'nonce': 766305500}]}, 'public_addr': '172.21.15.60:6821/766305500', 'cluster_addr': '172.21.15.60:6825/766305500', 'heartbeat_back_addr': '172.21.15.60:6831/766305500', 'heartbeat_front_addr': '172.21.15.60:6828/766305500', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f36c332c-9497-47e8-b943-f6b6dcaebeb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6812', 'nonce': 1032956039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6816', 'nonce': 1032956039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 1032956039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6822', 'nonce': 1032956039}]}, 'public_addr': '172.21.15.60:6812/1032956039', 'cluster_addr': '172.21.15.60:6816/1032956039', 'heartbeat_back_addr': '172.21.15.60:6829/1032956039', 'heartbeat_front_addr': '172.21.15.60:6822/1032956039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.183436+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.264080+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.253587+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.244429+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.369458+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.158089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.217295+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.332254+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.a', 'osds': [4, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:08:39.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:08:39.624 INFO:teuthology.orchestra.run.smithi031.stdout:3942 writesame 0xb978b6d thru 0xb978d67 data_size 0x3(0x1fb bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3943 write 0x249f5cf thru 0x24a0053 (0xa85 bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3944 trunc from 0xb978d68 to 0x8ace4ba 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3946 writesame 0x7a2e80f thru 0x7a2f69a data_size 0x4(0xe8c bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3948 read 0x6aa2eff thru 0x6aab111 (0x8213 bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3949 read 0xdda326 thru 0xddb355 (0x1030 bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3950 punch from 0x1efe1a3 to 0x1efefef, (0xe4c bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3951 read 0x1cc5e2d thru 0x1ccd2b3 (0x7487 bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3952 read 0x6b24499 thru 0x6b2b4d4 (0x703c bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3955 trunc from 0x8ace4ba to 0x6d19f85 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3957 read 0x201eeef thru 0x20290ab (0xa1bd bytes) 2024-09-13T09:08:39.625 INFO:teuthology.orchestra.run.smithi031.stdout:3959 read 0x2e36d32 thru 0x2e3813d (0x140c bytes) 2024-09-13T09:08:39.626 INFO:teuthology.orchestra.run.smithi031.stdout:3960 trunc from 0x6d19f85 to 0x5bed0c7 2024-09-13T09:08:39.626 INFO:teuthology.orchestra.run.smithi031.stdout:3961 punch from 0x1fa9bdc to 0x1fad439, (0x385d bytes) 2024-09-13T09:08:39.626 INFO:teuthology.orchestra.run.smithi031.stdout:3962 trunc from 0x5bed0c7 to 0x1b7fdb5 2024-09-13T09:08:39.626 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:39.617+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:39.647 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:39.647 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:39.647 INFO:teuthology.orchestra.run.smithi031.stdout:3963 read 0xe94234 thru 0xea06f2 (0xc4bf bytes) 2024-09-13T09:08:39.657 INFO:teuthology.orchestra.run.smithi031.stdout:3964 write 0x4117df1 thru 0x4122b2a (0xad3a bytes) 2024-09-13T09:08:39.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:39.675 INFO:teuthology.orchestra.run.smithi031.stdout:3965 read 0x1d6ec48 thru 0x1d7c7db (0xdb94 bytes) 2024-09-13T09:08:39.675 INFO:teuthology.orchestra.run.smithi031.stdout:3966 read 0x500e5c thru 0x50f040 (0xe1e5 bytes) 2024-09-13T09:08:39.683 INFO:teuthology.orchestra.run.smithi031.stdout:3967 write 0x5e9a51f thru 0x5ea8385 (0xde67 bytes) 2024-09-13T09:08:39.703 INFO:teuthology.orchestra.run.smithi031.stdout:3968 read 0x5aee1f9 thru 0x5af340c (0x5214 bytes) 2024-09-13T09:08:39.703 INFO:teuthology.orchestra.run.smithi031.stdout:3969 read 0x5c23f06 thru 0x5c28e66 (0x4f61 bytes) 2024-09-13T09:08:39.715 INFO:teuthology.orchestra.run.smithi031.stdout:3970 writesame 0x8f919f6 thru 0x8f9fece data_size 0x1(0xe4d9 bytes) 2024-09-13T09:08:39.722 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:39.719+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:39.727 INFO:teuthology.orchestra.run.smithi031.stdout:3971 write 0xd53c020 thru 0xd547238 (0xb219 bytes) 2024-09-13T09:08:39.742 INFO:teuthology.orchestra.run.smithi031.stdout:3972 read 0x43c35d5 thru 0x43cd585 (0x9fb1 bytes) 2024-09-13T09:08:39.743 INFO:teuthology.orchestra.run.smithi031.stdout:3973 read 0x69e00c6 thru 0x69e4a86 (0x49c1 bytes) 2024-09-13T09:08:39.743 INFO:teuthology.orchestra.run.smithi031.stdout:3974 punch from 0xadf3fca to 0xadf402c, (0x62 bytes) 2024-09-13T09:08:39.743 INFO:teuthology.orchestra.run.smithi031.stdout:3975 writesame 0x7f69802 thru 0x7f76efb data_size 0x7(0xd6fa bytes) 2024-09-13T09:08:39.766 INFO:teuthology.orchestra.run.smithi031.stdout:3978 writesame 0x443c044 thru 0x444b330 data_size 0x1(0xf2ed bytes) 2024-09-13T09:08:39.825 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:39.823+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:39.826 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:39.826 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:39.833 INFO:teuthology.orchestra.run.smithi031.stdout:3979 trunc from 0xd547239 to 0x34ae04a 2024-09-13T09:08:39.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:39.870 INFO:teuthology.orchestra.run.smithi031.stdout:3980 read 0x114dc7a thru 0x1152405 (0x478c bytes) 2024-09-13T09:08:39.870 INFO:teuthology.orchestra.run.smithi031.stdout:3981 read 0x13328aa thru 0x133cbcf (0xa326 bytes) 2024-09-13T09:08:39.871 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:39.886 INFO:teuthology.orchestra.run.smithi031.stdout:3983 writesame 0xc7ab496 thru 0xc7b825d data_size 0x3c(0xcdc8 bytes) 2024-09-13T09:08:39.895 INFO:teuthology.orchestra.run.smithi031.stdout:3984 write 0xe81b1b0 thru 0xe825772 (0xa5c3 bytes) 2024-09-13T09:08:39.912 INFO:teuthology.orchestra.run.smithi031.stdout:3986 writesame 0xdc60860 thru 0xdc6b79d data_size 0x3(0xaf3e bytes) 2024-09-13T09:08:39.926 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:39.921+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:39.931 INFO:teuthology.orchestra.run.smithi031.stdout:3987 write 0x1475467 thru 0x147b668 (0x6202 bytes) 2024-09-13T09:08:39.940 INFO:teuthology.orchestra.run.smithi031.stdout:3988 punch from 0xb05ba17 to 0xb06b96c, (0xff55 bytes) 2024-09-13T09:08:39.941 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:39.951 INFO:teuthology.orchestra.run.smithi031.stdout:3991 trunc from 0xe825773 to 0x4e6137f 2024-09-13T09:08:39.974 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:39.974 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:39.974 INFO:teuthology.orchestra.run.smithi031.stdout:3994 read 0x390bf9f thru 0x39109c0 (0x4a22 bytes) 2024-09-13T09:08:39.974 INFO:teuthology.orchestra.run.smithi031.stdout:3995 writesame 0x28659f2 thru 0x28665a0 data_size 0x1(0xbaf bytes) 2024-09-13T09:08:39.979 INFO:teuthology.orchestra.run.smithi031.stdout:3997 punch from 0x112f6a2 to 0x1132f1f, (0x387d bytes) 2024-09-13T09:08:39.983 INFO:teuthology.orchestra.run.smithi031.stdout:3999 trunc from 0x4e6137f to 0x18cb9b1 2024-09-13T09:08:39.990 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:39.991 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:08:40.006 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:40.006 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:40.015 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-13T09:08:39.205414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1168353634,"num_objects":969,"num_object_clones":202,"num_object_copies":1938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":162,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":969,"num_whiteouts":252,"num_read":140998,"num_read_kb":467173,"num_write":12105,"num_write_kb":252314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":475,"num_bytes_recovered":77798121,"num_keys_recovered":93,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9623,"ondisk_log_size":9623,"up":82,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":602880,"kb_used_data":300672,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754371840,"statfs":{"total":773094113280,"available":772476764160,"internally_reserved":0,"allocated":307888128,"data_stored":321795890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":309317176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46162534,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":162,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":7242,"num_read_kb":15047,"num_write":1809,"num_write_kb":4480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":959360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.034877"},"pg_stats":[{"pgid":"3.1f","version":"85'356","reported_seq":3353,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T09:08:37.196406+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:08:37.196406+0000","last_peered":"2024-09-13T09:08:37.196406+0000","last_clean":"2024-09-13T09:08:37.196406+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:08:37.196406+0000","last_undegraded":"2024-09-13T09:08:37.196406+0000","last_fullsized":"2024-09-13T09:08:37.196406+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":30293929,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":4013,"num_read_kb":12661,"num_write":507,"num_write_kb":3779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"85'399","reported_seq":2320,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T09:08:37.196565+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:08:37.196565+0000","last_peered":"2024-09-13T09:08:37.196565+0000","last_clean":"2024-09-13T09:08:37.196565+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:08:37.196565+0000","last_undegraded":"2024-09-13T09:08:37.196565+0000","last_fullsized":"2024-09-13T09:08:37.196565+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":43834390,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":8,"num_read":3678,"num_read_kb":10891,"num_write":480,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"85'777","reported_seq":3672,"reported_epoch":85,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-13T09:08:36.238833+0000","last_change":"2024-09-13T09:08:36.218256+0000","last_active":"2024-09-13T09:08:36.238833+0000","last_peered":"2024-09-13T09:08:36.238833+0000","last_clean":"2024-09-13T09:08:34.166180+0000","last_became_active":"2024-09-13T09:08:36.215259+0000","last_became_peered":"2024-09-13T09:08:36.215259+0000","last_unstale":"2024-09-13T09:08:36.238833+0000","last_undegraded":"2024-09-13T09:08:36.181973+0000","last_fullsized":"2024-09-13T09:08:36.179310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:27:02.015681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":54652883,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":35,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":11,"num_read":5771,"num_read_kb":29005,"num_write":1302,"num_write_kb":14512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":12314,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3],"avail_no_missing":["3","0"],"object_location_counts":[{"shards":"0,3","objects":35},{"shards":"3","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"82'295","reported_seq":2337,"reported_epoch":85,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:08:36.191921+0000","last_change":"2024-09-13T09:08:36.192418+0000","last_active":"2024-09-13T09:08:36.191921+0000","last_peered":"2024-09-13T09:08:36.191921+0000","last_clean":"2024-09-13T09:08:34.164938+0000","last_became_active":"2024-09-13T09:08:36.190848+0000","last_became_peered":"2024-09-13T09:08:36.190848+0000","last_unstale":"2024-09-13T09:08:36.191921+0000","last_undegraded":"2024-09-13T09:08:36.179042+0000","last_fullsized":"2024-09-13T09:08:36.177999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:23:48.432372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":67438462,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":7,"num_read":3942,"num_read_kb":25318,"num_write":381,"num_write_kb":11125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":3944384,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":30}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"82'383","reported_seq":20044,"reported_epoch":84,"state":"remapped+peering","last_fresh":"2024-09-13T09:08:35.179537+0000","last_change":"2024-09-13T09:08:35.180194+0000","last_active":"2024-09-13T09:08:34.172926+0000","last_peered":"2024-09-13T09:08:34.150977+0000","last_clean":"2024-09-13T09:08:34.150977+0000","last_became_active":"2024-09-13T09:07:13.862565+0000","last_became_peered":"2024-09-13T09:07:13.862565+0000","last_unstale":"2024-09-13T09:08:35.179537+0000","last_undegraded":"2024-09-13T09:08:35.179537+0000","last_fullsized":"2024-09-13T09:08:35.179537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:36:11.260401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":83732825,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":21605,"num_read_kb":39609,"num_write":452,"num_write_kb":17643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"85'339","reported_seq":2829,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T09:08:36.223790+0000","last_change":"2024-09-13T09:08:09.864519+0000","last_active":"2024-09-13T09:08:36.223790+0000","last_peered":"2024-09-13T09:08:36.223790+0000","last_clean":"2024-09-13T09:08:36.223790+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:08:36.223790+0000","last_undegraded":"2024-09-13T09:08:36.223790+0000","last_fullsized":"2024-09-13T09:08:36.223790+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":49078949,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":4253,"num_read_kb":14889,"num_write":505,"num_write_kb":16987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"82'143","reported_seq":871,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.171467+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:08:34.171467+0000","last_peered":"2024-09-13T09:08:34.171467+0000","last_clean":"2024-09-13T09:08:34.171467+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:08:34.171467+0000","last_undegraded":"2024-09-13T09:08:34.171467+0000","last_fullsized":"2024-09-13T09:08:34.171467+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":4046418,"num_objects":25,"num_object_clones":6,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":6,"num_read":645,"num_read_kb":4984,"num_write":142,"num_write_kb":6081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"85'402","reported_seq":1692,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T09:08:37.195790+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:08:37.195790+0000","last_peered":"2024-09-13T09:08:37.195790+0000","last_clean":"2024-09-13T09:08:37.195790+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:08:37.195790+0000","last_undegraded":"2024-09-13T09:08:37.195790+0000","last_fullsized":"2024-09-13T09:08:37.195790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":42090000,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":7,"num_read":2651,"num_read_kb":9185,"num_write":490,"num_write_kb":6731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"85'320","reported_seq":4024,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T09:08:37.196485+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:08:37.196485+0000","last_peered":"2024-09-13T09:08:37.196485+0000","last_clean":"2024-09-13T09:08:37.196485+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:08:37.196485+0000","last_undegraded":"2024-09-13T09:08:37.196485+0000","last_fullsized":"2024-09-13T09:08:37.196485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":49418563,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":4694,"num_read_kb":25732,"num_write":344,"num_write_kb":16617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"82'189","reported_seq":772,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.172380+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:08:35.172380+0000","last_peered":"2024-09-13T09:08:35.172380+0000","last_clean":"2024-09-13T09:08:35.172380+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:08:35.172380+0000","last_undegraded":"2024-09-13T09:08:35.172380+0000","last_fullsized":"2024-09-13T09:08:35.172380+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":29364381,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":709,"num_read_kb":8419,"num_write":199,"num_write_kb":5848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"82'333","reported_seq":2710,"reported_epoch":84,"state":"remapped+peering","last_fresh":"2024-09-13T09:08:35.187128+0000","last_change":"2024-09-13T09:08:35.187128+0000","last_active":"2024-09-13T09:08:34.178754+0000","last_peered":"2024-09-13T09:08:34.155735+0000","last_clean":"2024-09-13T09:08:34.155735+0000","last_became_active":"2024-09-13T09:07:13.866813+0000","last_became_peered":"2024-09-13T09:07:13.866813+0000","last_unstale":"2024-09-13T09:08:35.187128+0000","last_undegraded":"2024-09-13T09:08:35.187128+0000","last_fullsized":"2024-09-13T09:08:35.187128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:46.897544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":57077406,"num_objects":38,"num_object_clones":12,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":11,"num_read":3426,"num_read_kb":18639,"num_write":328,"num_write_kb":12313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"85'358","reported_seq":17491,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T09:08:36.180051+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:08:36.180051+0000","last_peered":"2024-09-13T09:08:36.180051+0000","last_clean":"2024-09-13T09:08:36.180051+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:08:36.180051+0000","last_undegraded":"2024-09-13T09:08:36.180051+0000","last_fullsized":"2024-09-13T09:08:36.180051+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":32755134,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":7,"num_read":18921,"num_read_kb":29668,"num_write":328,"num_write_kb":8957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.189144+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:08:35.189144+0000","last_peered":"2024-09-13T09:08:35.189144+0000","last_clean":"2024-09-13T09:08:35.189144+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:08:35.189144+0000","last_undegraded":"2024-09-13T09:08:35.189144+0000","last_fullsized":"2024-09-13T09:08:35.189144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"82'430","reported_seq":3265,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.171608+0000","last_change":"2024-09-13T09:07:16.108420+0000","last_active":"2024-09-13T09:08:34.171608+0000","last_peered":"2024-09-13T09:08:34.171608+0000","last_clean":"2024-09-13T09:08:34.171608+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:08:34.171608+0000","last_undegraded":"2024-09-13T09:08:34.171608+0000","last_fullsized":"2024-09-13T09:08:34.171608+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":32561741,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":6,"num_read":4230,"num_read_kb":9027,"num_write":534,"num_write_kb":2300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":87,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.184636+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:08:35.184636+0000","last_peered":"2024-09-13T09:08:35.184636+0000","last_clean":"2024-09-13T09:08:35.184636+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:08:35.184636+0000","last_undegraded":"2024-09-13T09:08:35.184636+0000","last_fullsized":"2024-09-13T09:08:35.184636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"82'235","reported_seq":1633,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.171704+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:08:34.171704+0000","last_peered":"2024-09-13T09:08:34.171704+0000","last_clean":"2024-09-13T09:08:34.171704+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:08:34.171704+0000","last_undegraded":"2024-09-13T09:08:34.171704+0000","last_fullsized":"2024-09-13T09:08:34.171704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":26259712,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":2167,"num_read_kb":12707,"num_write":231,"num_write_kb":4568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":80,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.173660+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:08:35.173660+0000","last_peered":"2024-09-13T09:08:35.173660+0000","last_clean":"2024-09-13T09:08:35.173660+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:08:35.173660+0000","last_undegraded":"2024-09-13T09:08:35.173660+0000","last_fullsized":"2024-09-13T09:08:35.173660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"82'315","reported_seq":3367,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.178012+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:08:34.178012+0000","last_peered":"2024-09-13T09:08:34.178012+0000","last_clean":"2024-09-13T09:08:34.178012+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:08:34.178012+0000","last_undegraded":"2024-09-13T09:08:34.178012+0000","last_fullsized":"2024-09-13T09:08:34.178012+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":37265822,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":3962,"num_read_kb":14191,"num_write":471,"num_write_kb":2959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T09:08:37.196369+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:08:37.196369+0000","last_peered":"2024-09-13T09:08:37.196369+0000","last_clean":"2024-09-13T09:08:37.196369+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:08:37.196369+0000","last_undegraded":"2024-09-13T09:08:37.196369+0000","last_fullsized":"2024-09-13T09:08:37.196369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"82'297","reported_seq":2315,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.172648+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:08:35.172648+0000","last_peered":"2024-09-13T09:08:35.172648+0000","last_clean":"2024-09-13T09:08:35.172648+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:08:35.172648+0000","last_undegraded":"2024-09-13T09:08:35.172648+0000","last_fullsized":"2024-09-13T09:08:35.172648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":27270000,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":2921,"num_read_kb":14469,"num_write":432,"num_write_kb":9601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"82'183","reported_seq":1250,"reported_epoch":85,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:08:36.189365+0000","last_change":"2024-09-13T09:08:36.189365+0000","last_active":"2024-09-13T09:08:36.189365+0000","last_peered":"2024-09-13T09:08:36.189365+0000","last_clean":"2024-09-13T09:08:33.811372+0000","last_became_active":"2024-09-13T09:08:36.188785+0000","last_became_peered":"2024-09-13T09:08:36.188785+0000","last_unstale":"2024-09-13T09:08:36.189365+0000","last_undegraded":"2024-09-13T09:08:36.174308+0000","last_fullsized":"2024-09-13T09:08:36.173584+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:32.968805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":45087964,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":51,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":1627,"num_read_kb":7582,"num_write":194,"num_write_kb":1626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":30}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":82,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.173458+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:08:35.173458+0000","last_peered":"2024-09-13T09:08:35.173458+0000","last_clean":"2024-09-13T09:08:35.173458+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:08:35.173458+0000","last_undegraded":"2024-09-13T09:08:35.173458+0000","last_fullsized":"2024-09-13T09:08:35.173458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"85'196","reported_seq":1697,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T09:08:37.195506+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:08:37.195506+0000","last_peered":"2024-09-13T09:08:37.195506+0000","last_clean":"2024-09-13T09:08:37.195506+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:08:37.195506+0000","last_undegraded":"2024-09-13T09:08:37.195506+0000","last_fullsized":"2024-09-13T09:08:37.195506+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":13739477,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":1506,"num_read_kb":10475,"num_write":194,"num_write_kb":5354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":79,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.174126+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:08:35.174126+0000","last_peered":"2024-09-13T09:08:35.174126+0000","last_clean":"2024-09-13T09:08:35.174126+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:08:35.174126+0000","last_undegraded":"2024-09-13T09:08:35.174126+0000","last_fullsized":"2024-09-13T09:08:35.174126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'228","reported_seq":1236,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.177828+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:08:34.177828+0000","last_peered":"2024-09-13T09:08:34.177828+0000","last_clean":"2024-09-13T09:08:34.177828+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:08:34.177828+0000","last_undegraded":"2024-09-13T09:08:34.177828+0000","last_fullsized":"2024-09-13T09:08:34.177828+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":14258824,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":1319,"num_read_kb":10997,"num_write":225,"num_write_kb":5871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":80,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.173092+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:08:35.173092+0000","last_peered":"2024-09-13T09:08:35.173092+0000","last_clean":"2024-09-13T09:08:35.173092+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:08:35.173092+0000","last_undegraded":"2024-09-13T09:08:35.173092+0000","last_fullsized":"2024-09-13T09:08:35.173092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"82'232","reported_seq":581,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.171826+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:08:34.171826+0000","last_peered":"2024-09-13T09:08:34.171826+0000","last_clean":"2024-09-13T09:08:34.171826+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:08:34.171826+0000","last_undegraded":"2024-09-13T09:08:34.171826+0000","last_fullsized":"2024-09-13T09:08:34.171826+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":7839411,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":893,"num_read_kb":4516,"num_write":269,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":81,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.173360+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:08:35.173360+0000","last_peered":"2024-09-13T09:08:35.173360+0000","last_clean":"2024-09-13T09:08:35.173360+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:08:35.173360+0000","last_undegraded":"2024-09-13T09:08:35.173360+0000","last_fullsized":"2024-09-13T09:08:35.173360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":210,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.178260+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:08:35.178260+0000","last_peered":"2024-09-13T09:08:35.178260+0000","last_clean":"2024-09-13T09:08:35.178260+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:08:35.178260+0000","last_undegraded":"2024-09-13T09:08:35.178260+0000","last_fullsized":"2024-09-13T09:08:35.178260+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"82'247","reported_seq":3862,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.171369+0000","last_change":"2024-09-13T09:07:16.108021+0000","last_active":"2024-09-13T09:08:34.171369+0000","last_peered":"2024-09-13T09:08:34.171369+0000","last_clean":"2024-09-13T09:08:34.171369+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:08:34.171369+0000","last_undegraded":"2024-09-13T09:08:34.171369+0000","last_fullsized":"2024-09-13T09:08:34.171369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":56271714,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":9,"num_read":3937,"num_read_kb":19169,"num_write":392,"num_write_kb":8952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"82'283","reported_seq":2202,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.173224+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:08:35.173224+0000","last_peered":"2024-09-13T09:08:35.173224+0000","last_clean":"2024-09-13T09:08:35.173224+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:08:35.173224+0000","last_undegraded":"2024-09-13T09:08:35.173224+0000","last_fullsized":"2024-09-13T09:08:35.173224+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":42719238,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":4,"num_read":3010,"num_read_kb":10707,"num_write":359,"num_write_kb":5267,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"83'300","reported_seq":1369,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.175812+0000","last_change":"2024-09-13T09:08:21.089403+0000","last_active":"2024-09-13T09:08:34.175812+0000","last_peered":"2024-09-13T09:08:34.175812+0000","last_clean":"2024-09-13T09:08:34.175812+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:08:34.175812+0000","last_undegraded":"2024-09-13T09:08:34.175812+0000","last_fullsized":"2024-09-13T09:08:34.175812+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":48400653,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":1752,"num_read_kb":15218,"num_write":299,"num_write_kb":16332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"85'134","reported_seq":1438,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T09:08:36.179535+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:08:36.179535+0000","last_peered":"2024-09-13T09:08:36.179535+0000","last_clean":"2024-09-13T09:08:36.179535+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:08:36.179535+0000","last_undegraded":"2024-09-13T09:08:36.179535+0000","last_fullsized":"2024-09-13T09:08:36.179535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":16867764,"num_objects":27,"num_object_clones":8,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":1299,"num_read_kb":19311,"num_write":162,"num_write_kb":13466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"82'168","reported_seq":16055,"reported_epoch":85,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-13T09:08:36.190499+0000","last_change":"2024-09-13T09:08:36.190499+0000","last_active":"2024-09-13T09:08:36.190499+0000","last_peered":"2024-09-13T09:08:36.190499+0000","last_clean":"2024-09-13T09:08:33.576022+0000","last_became_active":"2024-09-13T09:08:36.189474+0000","last_became_peered":"2024-09-13T09:08:36.189474+0000","last_unstale":"2024-09-13T09:08:36.190499+0000","last_undegraded":"2024-09-13T09:08:36.178597+0000","last_fullsized":"2024-09-13T09:08:36.177551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:36:12.664440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":48361532,"num_objects":34,"num_object_clones":10,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":61,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":10,"num_read":16175,"num_read_kb":25673,"num_write":208,"num_write_kb":13461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7],"avail_no_missing":["7","0"],"object_location_counts":[{"shards":"0,7","objects":34}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"82'176","reported_seq":2090,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.173560+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:08:35.173560+0000","last_peered":"2024-09-13T09:08:35.173560+0000","last_clean":"2024-09-13T09:08:35.173560+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:08:35.173560+0000","last_undegraded":"2024-09-13T09:08:35.173560+0000","last_fullsized":"2024-09-13T09:08:35.173560+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":21716089,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":2413,"num_read_kb":6567,"num_write":175,"num_write_kb":3861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"82'215","reported_seq":4207,"reported_epoch":85,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-13T09:08:36.223385+0000","last_change":"2024-09-13T09:08:36.189225+0000","last_active":"2024-09-13T09:08:36.223385+0000","last_peered":"2024-09-13T09:08:36.223385+0000","last_clean":"2024-09-13T09:08:34.153858+0000","last_became_active":"2024-09-13T09:08:36.185551+0000","last_became_peered":"2024-09-13T09:08:36.185551+0000","last_unstale":"2024-09-13T09:08:36.223385+0000","last_undegraded":"2024-09-13T09:08:36.223385+0000","last_fullsized":"2024-09-13T09:08:36.174404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:56:01.743934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":31994397,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":21,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":11,"num_read":4449,"num_read_kb":10352,"num_write":248,"num_write_kb":2318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":890690,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":21},{"shards":"2","objects":12}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"82'171","reported_seq":2709,"reported_epoch":85,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-13T09:08:36.230077+0000","last_change":"2024-09-13T09:08:36.193209+0000","last_active":"2024-09-13T09:08:36.230077+0000","last_peered":"2024-09-13T09:08:36.230077+0000","last_clean":"2024-09-13T09:08:34.009834+0000","last_became_active":"2024-09-13T09:08:36.189414+0000","last_became_peered":"2024-09-13T09:08:36.189414+0000","last_unstale":"2024-09-13T09:08:36.230077+0000","last_undegraded":"2024-09-13T09:08:36.230077+0000","last_fullsized":"2024-09-13T09:08:36.178000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":171,"log_dups_size":0,"ondisk_log_size":171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:41:50.167137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":20874027,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":2790,"num_read_kb":7654,"num_write":170,"num_write_kb":3174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":941886,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1],"avail_no_missing":["1","0"],"object_location_counts":[{"shards":"0,1","objects":17},{"shards":"1","objects":10}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"85'224","reported_seq":1759,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T09:08:36.180605+0000","last_change":"2024-09-13T09:08:09.863895+0000","last_active":"2024-09-13T09:08:36.180605+0000","last_peered":"2024-09-13T09:08:36.180605+0000","last_clean":"2024-09-13T09:08:36.180605+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:08:36.180605+0000","last_undegraded":"2024-09-13T09:08:36.180605+0000","last_fullsized":"2024-09-13T09:08:36.180605+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":54641299,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":7,"num_read":1496,"num_read_kb":14071,"num_write":236,"num_write_kb":8048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"82'627","reported_seq":2898,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.173146+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:08:35.173146+0000","last_peered":"2024-09-13T09:08:35.173146+0000","last_clean":"2024-09-13T09:08:35.173146+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:08:35.173146+0000","last_undegraded":"2024-09-13T09:08:35.173146+0000","last_fullsized":"2024-09-13T09:08:35.173146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":26420943,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":6,"num_read":5198,"num_read_kb":9446,"num_write":975,"num_write_kb":4332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"82'256","reported_seq":2604,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T09:08:34.177853+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:08:34.177853+0000","last_peered":"2024-09-13T09:08:34.177853+0000","last_clean":"2024-09-13T09:08:34.177853+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:08:34.177853+0000","last_undegraded":"2024-09-13T09:08:34.177853+0000","last_fullsized":"2024-09-13T09:08:34.177853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":34510823,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":8,"num_read":3375,"num_read_kb":8519,"num_write":347,"num_write_kb":5057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"82'292","reported_seq":1435,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T09:08:35.173263+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:08:35.173263+0000","last_peered":"2024-09-13T09:08:35.173263+0000","last_clean":"2024-09-13T09:08:35.173263+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:08:35.173263+0000","last_undegraded":"2024-09-13T09:08:35.173263+0000","last_fullsized":"2024-09-13T09:08:35.173263+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":16918477,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":2055,"num_read_kb":7422,"num_write":365,"num_write_kb":6000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1167763247,"num_objects":965,"num_object_clones":202,"num_object_copies":1930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":162,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":965,"num_whiteouts":252,"num_read":140882,"num_read_kb":467073,"num_write":11944,"num_write_kb":249896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":473,"num_bytes_recovered":77207753,"num_keys_recovered":93,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":302686208,"data_stored":320863494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9503,"ondisk_log_size":9503,"up":64,"acting":56,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70416,"kb_used_data":31104,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94301424,"statfs":{"total":96636764160,"available":96564658176,"internally_reserved":0,"allocated":31850496,"data_stored":33278698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.83299999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.75900000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.58599999999999997}]}]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81012,"kb_used_data":41704,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94290828,"statfs":{"total":96636764160,"available":96553807872,"internally_reserved":0,"allocated":42704896,"data_stored":46112862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":40237512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.443}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.96699999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.78600000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75552,"kb_used_data":36240,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94296288,"statfs":{"total":96636764160,"available":96559398912,"internally_reserved":0,"allocated":37109760,"data_stored":37319160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.435}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.497}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.746},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.69999999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.622},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.89500000000000002}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.88100000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67240,"kb_used_data":32024,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94304600,"statfs":{"total":96636764160,"available":96567910400,"internally_reserved":0,"allocated":32792576,"data_stored":32865034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.34599999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.98399999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.96799999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.84399999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86992,"kb_used_data":47668,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94284848,"statfs":{"total":96636764160,"available":96547684352,"internally_reserved":0,"allocated":48812032,"data_stored":54151842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.498},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.41799999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":3.0569999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.87}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":2.1629999999999998},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.82799999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.68500000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542167,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68008,"kb_used_data":28700,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94303832,"statfs":{"total":96636764160,"available":96567123968,"internally_reserved":0,"allocated":29388800,"data_stored":33174877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.81200000000000006}]},{"osd":2,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.752},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.625}]},{"osd":7,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.72799999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75788,"kb_used_data":32384,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94296052,"statfs":{"total":96636764160,"available":96559157248,"internally_reserved":0,"allocated":33161216,"data_stored":33550975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.68899999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.0329999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77872,"kb_used_data":50848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94293968,"statfs":{"total":96636764160,"available":96557023232,"internally_reserved":0,"allocated":52068352,"data_stored":51342442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.9740000000000002,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.9050000000000002,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.70799999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":8.0850000000000009,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.48899999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.58799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":47828992,"data_stored":53702904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31236096,"data_stored":31842258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36126720,"data_stored":36870222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45531136,"data_stored":49462249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":32190464,"data_stored":33108319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":50483200,"data_stored":50307388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28413952,"data_stored":32732202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":30875648,"data_stored":32837952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:08:40.018 INFO:tasks.thrashosds.thrasher:Setting 3.f to [3, 6, 1, 0] 2024-09-13T09:08:40.019 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.f', '3', '6', '1', '0'] 2024-09-13T09:08:40.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.f 3 6 1 0 2024-09-13T09:08:40.020 INFO:teuthology.orchestra.run.smithi031.stdout:4000 write 0xd6bd162 thru 0xd6c7c76 (0xab15 bytes) 2024-09-13T09:08:40.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:40.030 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:40.027+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:40.041 INFO:teuthology.orchestra.run.smithi031.stdout:4001 read 0xa72a3ad thru 0xa732db7 (0x8a0b bytes) 2024-09-13T09:08:40.041 INFO:teuthology.orchestra.run.smithi031.stdout:4002 write 0xaaf9733 thru 0xab063e8 (0xccb6 bytes) 2024-09-13T09:08:40.075 INFO:teuthology.orchestra.run.smithi031.stdout:4003 trunc from 0xd6c7c77 to 0x22f831d 2024-09-13T09:08:40.096 INFO:teuthology.orchestra.run.smithi031.stdout:4004 write 0x1e5f3c6 thru 0x1e6cb7f (0xd7ba bytes) 2024-09-13T09:08:40.130 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:40.127+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:40.134 INFO:teuthology.orchestra.run.smithi031.stdout:4006 writesame 0xc6a76d0 thru 0xc6a961a data_size 0x1(0x1f4b bytes) 2024-09-13T09:08:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:40.135 INFO:teuthology.orchestra.run.smithi031.stdout:4008 write 0x16980ca thru 0x16a4f13 (0xce4a bytes) 2024-09-13T09:08:40.162 INFO:teuthology.orchestra.run.smithi031.stdout:4009 trunc from 0xc6a961b to 0x25c5358 2024-09-13T09:08:40.179 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:40.179 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:40.196 INFO:teuthology.orchestra.run.smithi031.stdout:4010 write 0xc7ecf5e thru 0xc7f6ae4 (0x9b87 bytes) 2024-09-13T09:08:40.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:40.215 INFO:teuthology.orchestra.run.smithi031.stdout:4011 read 0x2e9af8 thru 0x2f8b50 (0xf059 bytes) 2024-09-13T09:08:40.217 INFO:teuthology.orchestra.run.smithi031.stdout:4012 writesame 0x66cb87e thru 0x66d684e data_size 0x9(0xafd1 bytes) 2024-09-13T09:08:40.225 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:08:40.238 INFO:teuthology.orchestra.run.smithi031.stdout:4014 write 0x61a285c thru 0x61a5ca4 (0x3449 bytes) 2024-09-13T09:08:40.242 INFO:teuthology.orchestra.run.smithi031.stdout:4015 read 0xa0c16a5 thru 0xa0d11e4 (0xfb40 bytes) 2024-09-13T09:08:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:4016 read 0x43d07fa thru 0x43d612d (0x5934 bytes) 2024-09-13T09:08:40.243 INFO:teuthology.orchestra.run.smithi031.stdout:4018 read 0xa7dccd8 thru 0xa7ec62f (0xf958 bytes) 2024-09-13T09:08:40.249 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:40.245+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:40.257 INFO:teuthology.orchestra.run.smithi031.stdout:4019 trunc from 0xc7f6ae5 to 0x453d50 2024-09-13T09:08:40.331 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:40.325+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:40.346 INFO:teuthology.orchestra.run.smithi031.stdout:4020 read 0x26de72 thru 0x2757a9 (0x7938 bytes) 2024-09-13T09:08:40.348 INFO:teuthology.orchestra.run.smithi031.stdout:4022 read 0xfd930 thru 0x10c774 (0xee45 bytes) 2024-09-13T09:08:40.349 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:40.358 INFO:teuthology.orchestra.run.smithi031.stdout:4025 trunc from 0x453d50 to 0x8682e3a 2024-09-13T09:08:40.369 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:40.369 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:40.377 INFO:teuthology.orchestra.run.smithi031.stdout:4026 writesame 0xee65ba1 thru 0xee6b27f data_size 0x15(0x56df bytes) 2024-09-13T09:08:40.377 INFO:teuthology.orchestra.run.smithi031.stdout:4027 read 0x28001dc thru 0x2805206 (0x502b bytes) 2024-09-13T09:08:40.377 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:40.377 INFO:teuthology.orchestra.run.smithi031.stdout:4029 writesame 0x599d646 thru 0x59a8e4e data_size 0x1(0xb809 bytes) 2024-09-13T09:08:40.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:40.411 INFO:teuthology.orchestra.run.smithi031.stdout:4030 writesame 0x530542e thru 0x53089f2 data_size 0x1(0x35c5 bytes) 2024-09-13T09:08:40.423 INFO:teuthology.orchestra.run.smithi031.stdout:4031 punch from 0x939bf01 to 0x93a047d, (0x457c bytes) 2024-09-13T09:08:40.423 INFO:teuthology.orchestra.run.smithi031.stdout:4032 writesame 0x72f72d9 thru 0x730667c data_size 0x3e(0xf3a4 bytes) 2024-09-13T09:08:40.432 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:40.431+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:40.441 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:40.441 INFO:teuthology.orchestra.run.smithi031.stdout:4034 read 0x606ec58 thru 0x607d27c (0xe625 bytes) 2024-09-13T09:08:40.447 INFO:teuthology.orchestra.run.smithi031.stdout:4035 trunc from 0xee6b280 to 0xa62a445 2024-09-13T09:08:40.455 INFO:teuthology.orchestra.run.smithi031.stdout:4036 writesame 0x1eaa402 thru 0x1eae7c4 data_size 0x13(0x43c3 bytes) 2024-09-13T09:08:40.463 INFO:teuthology.orchestra.run.smithi031.stdout:4037 read 0x120f796 thru 0x1219c29 (0xa494 bytes) 2024-09-13T09:08:40.463 INFO:teuthology.orchestra.run.smithi031.stdout:4041 read 0x499e6c6 thru 0x49abd0e (0xd649 bytes) 2024-09-13T09:08:40.463 INFO:teuthology.orchestra.run.smithi031.stdout:4042 read 0x66384b2 thru 0x663aabf (0x260e bytes) 2024-09-13T09:08:40.463 INFO:teuthology.orchestra.run.smithi031.stdout:4043 clone 31 order 25 su 16777216 sc 2 2024-09-13T09:08:40.533 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:40.531+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:40.553 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:40.553 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:40.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:40.633 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:40.629+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:40.694 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:40.694 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:40.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:40.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:40.771 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:40.771+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:40.835 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:40.829+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:40.878 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:40.878 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:40.879 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:40.879 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:40.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:40.894 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:40.894 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:40.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:40.910 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:40.927 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:40.927 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:40.937 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:40.933+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:40.946 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:40.947 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:40.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:40.983 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:40.984 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:41.002 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:41.002 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:41.037 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:41.033+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:41.065 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:41.066 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:41.066 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:41.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:41.169 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:41.165+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:41.228 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.f pg_upmap_items mapping to [3->6,1->0] 2024-09-13T09:08:41.238 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:41.233+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.634869768000001, 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:08:41.257 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:41.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:08:41.261 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.634846437999997, 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.262 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:41.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.634823420999993, 2024-09-13T09:08:41.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:08:41.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:08:41.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.634800291000005, 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:08:41.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.632183527999999, 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.280 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.632159442000003, 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:08:41.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:08:41.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.632132788999996, 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:08:41.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:08:41.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.632109149000001, 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:08:41.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.628293608000007, 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:08:41.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.628270064999995, 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:08:41.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:08:41.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:08:41.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.628246774999994, 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:08:41.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:08:41.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.308 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.628223262000006, 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:08:41.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.628164806000001, 2024-09-13T09:08:41.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:41.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:08:41.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.624133869999994, 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:08:41.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:41.322 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.304235573, 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:08:41.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.301863793999999, 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:08:41.327 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:08:41.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:08:41.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.301329879000001, 2024-09-13T09:08:41.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:41.337 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.297929355999997, 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:08:41.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.342 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.296929548999998, 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:41.347 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 48.293759270999999, 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:41.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:08:41.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.352 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:41.353 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone30 intact 2024-09-13T09:08:41.354 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:41.355 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:41.351+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:41.457 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:41.455+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:41.503 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-13T09:08:41.532 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:41.532 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:41.532 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:41.532 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:41.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:41.597 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:41.593+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:41.658 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:41.653+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:41.673 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:41.673 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:41.673 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:41.673 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:41.673 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:41.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:41.760 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:41.759+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:41.855 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412799500000006, 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.856 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:41.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:08:41.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.860 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412774916999993, 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:08:41.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:08:41.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:41.865 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412750912999996, 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:41.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:08:41.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412727430000004, 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.870 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:41.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412704023000003, 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.875 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:08:41.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.879 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412680773999995, 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.884 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412648668000003, 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:08:41.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412613050999994, 2024-09-13T09:08:41.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:08:41.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.893 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412581071000005, 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.894 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:41.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:08:41.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:41.898 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412511792000004, 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:41.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:08:41.902 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412486732000005, 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.903 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:08:41.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.907 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412463302000006, 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:08:41.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:08:41.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412439918999993, 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.912 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:08:41.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.916 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412416503000003, 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:08:41.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412392580000002, 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.921 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:08:41.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412359495999993, 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:08:41.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412335696, 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.930 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:08:41.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:41.934 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412311369999998, 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:08:41.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.938 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412278810000004, 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.939 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:08:41.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 66.412238936999998, 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:41.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:41.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:41.947 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:41.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:41.949 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:41.945+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:42.050 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:42.045+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:42.125 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:42.125 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:42.125 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:42.125 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:42.139 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:42.181 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:42.177+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:42.252 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:08:42.252 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:42.251+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:42.303 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:42.303 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:42.303 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:42.303 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:42.303 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:42.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:42.365 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:42.361+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:42.442 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.442 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216987754000002, 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:42.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:08:42.446 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216956904, 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.447 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216930736999998, 2024-09-13T09:08:42.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:08:42.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:08:42.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216891262999994, 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.456 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:42.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:08:42.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216856708999998, 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.461 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.465 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216824353000007, 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:08:42.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:08:42.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:42.470 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216792081999998, 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:42.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:08:42.474 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216758045999995, 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.475 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:08:42.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:08:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:08:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:08:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216724782, 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.482 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.216692542000004, 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:08:42.487 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.492 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.207712455999996, 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.493 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:08:42.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:08:42.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.497 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.207671575999996, 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:08:42.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:08:42.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.207631308000003, 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:08:42.503 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:42.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:08:42.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.207584264999994, 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.509 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:42.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:08:42.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:08:42.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.514 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.207538702999997, 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:42.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:42.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:08:42.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:08:42.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.520 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.207448534999997, 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.207401981000004, 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:08:42.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:08:42.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:08:42.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:08:42.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.205761057000004, 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.531 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:08:42.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:42.536 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.205695250999995, 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:08:42.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 9.5226876059999999, 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:42.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:42.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:08:42.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:08:42.546 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:08:42.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:08:42.548 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.549 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.549 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:42.549 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:42.549 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:42.551 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:42.545+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:42.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:42.651 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:42.651+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:42.678 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:42.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:42.679 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:42.679 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:42.693 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:42.779 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:42.779+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:42.853 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:42.851+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:42.853 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:42.854 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:42.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:42.954 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:42.949+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:43.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.920158028000003, 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:08:43.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:08:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.920137261000001, 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:08:43.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.920119409000002, 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:08:43.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:08:43.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.920102843000002, 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:08:43.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:08:43.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.920077257999999, 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:08:43.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.920058091000001, 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:43.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.920040808000003, 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:08:43.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.919511937999999, 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:08:43.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:08:43.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.919480032999999, 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:08:43.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:08:43.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.919455085000003, 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:43.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:08:43.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.919150520000002, 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:08:43.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.917388097, 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:43.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:08:43.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:08:43.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:08:43.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.917367247000001, 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:08:43.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.917344923000002, 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:43.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.917326234000001, 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:08:43.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.917309181999997, 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:08:43.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.917292158000002, 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:08:43.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.907279170999999, 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:43.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:43.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:08:43.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.122 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.527124648, 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:08:43.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:08:43.127 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5269442259999999, 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:43.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:08:43.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:08:43.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:43.132 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:43.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:43.134 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:43.131+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:43.175 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:43.175 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:43.193 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:43.193 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:43.209 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:43.209 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:43.226 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:43.226 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:43.234 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:43.231+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:43.247 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:43.247 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:43.252 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:08:43.270 INFO:teuthology.orchestra.run.smithi031.stdout:4044 punch from 0x6cf7f42 to 0x6cfed13, (0x6dd1 bytes) 2024-09-13T09:08:43.277 INFO:teuthology.orchestra.run.smithi031.stdout:4047 write 0x6809e84 thru 0x6811003 (0x7180 bytes) 2024-09-13T09:08:43.278 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:43.279 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:43.298 INFO:teuthology.orchestra.run.smithi031.stdout:4048 trunc from 0xa62a445 to 0xe1db595 2024-09-13T09:08:43.307 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:43.307 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:43.313 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:43.313 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:43.313 INFO:teuthology.orchestra.run.smithi031.stdout:4050 trunc from 0xe1db595 to 0x9be635d 2024-09-13T09:08:43.330 INFO:teuthology.orchestra.run.smithi031.stdout:4051 trunc from 0x9be635d to 0x91e1914 2024-09-13T09:08:43.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:43.334 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:43.334 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:43.335 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:43.331+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:43.343 INFO:teuthology.orchestra.run.smithi031.stdout:4052 trunc from 0x91e1914 to 0xb977b47 2024-09-13T09:08:43.351 INFO:teuthology.orchestra.run.smithi031.stdout:4053 read 0x344a5a3 thru 0x3451516 (0x6f74 bytes) 2024-09-13T09:08:43.359 INFO:teuthology.orchestra.run.smithi031.stdout:4054 writesame 0xba328a7 thru 0xba42729 data_size 0x53(0xfe83 bytes) 2024-09-13T09:08:43.370 INFO:teuthology.orchestra.run.smithi031.stdout:4055 trunc from 0xba4272a to 0x4c42d28 2024-09-13T09:08:43.402 INFO:teuthology.orchestra.run.smithi031.stdout:4056 write 0xc1b5ea8 thru 0xc1bab0e (0x4c67 bytes) 2024-09-13T09:08:43.408 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:43.408 INFO:teuthology.orchestra.run.smithi031.stdout:4058 read 0x421d80c thru 0x422601b (0x8810 bytes) 2024-09-13T09:08:43.409 INFO:teuthology.orchestra.run.smithi031.stdout:4059 write 0xba82ec5 thru 0xba8adf6 (0x7f32 bytes) 2024-09-13T09:08:43.413 INFO:teuthology.orchestra.run.smithi031.stdout:4061 writesame 0x4918f20 thru 0x4924d13 data_size 0x2(0xbdf4 bytes) 2024-09-13T09:08:43.428 INFO:teuthology.orchestra.run.smithi031.stdout:4062 writesame 0x353648e thru 0x3538f55 data_size 0x8(0x2ac8 bytes) 2024-09-13T09:08:43.436 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:43.429+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:43.437 INFO:teuthology.orchestra.run.smithi031.stdout:4063 punch from 0xb944954 to 0xb95391a, (0xefc6 bytes) 2024-09-13T09:08:43.437 INFO:teuthology.orchestra.run.smithi031.stdout:4064 writesame 0x59be4a1 thru 0x59c904e data_size 0x4b(0xabae bytes) 2024-09-13T09:08:43.441 INFO:teuthology.orchestra.run.smithi031.stdout:4065 write 0xc12aa8f thru 0xc12e4ff (0x3a71 bytes) 2024-09-13T09:08:43.445 INFO:teuthology.orchestra.run.smithi031.stdout:4067 write 0x5f87898 thru 0x5f8d1d2 (0x593b bytes) 2024-09-13T09:08:43.447 INFO:teuthology.orchestra.run.smithi031.stdout:4068 read 0x5ef6cc5 thru 0x5efca36 (0x5d72 bytes) 2024-09-13T09:08:43.448 INFO:teuthology.orchestra.run.smithi031.stdout:4070 punch from 0x194fd64 to 0x195c613, (0xc8af bytes) 2024-09-13T09:08:43.448 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:43.448 INFO:teuthology.orchestra.run.smithi031.stdout:4075 punch from 0x5c3ab87 to 0x5c47ef7, (0xd370 bytes) 2024-09-13T09:08:43.449 INFO:teuthology.orchestra.run.smithi031.stdout:4076 write 0x673a8f1 thru 0x67482c2 (0xd9d2 bytes) 2024-09-13T09:08:43.451 INFO:teuthology.orchestra.run.smithi031.stdout:4077 write 0xf0f464 thru 0xf12747 (0x32e4 bytes) 2024-09-13T09:08:43.468 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:43.469 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:43.482 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:43.508 INFO:teuthology.orchestra.run.smithi031.stdout:4078 trunc from 0xc1bab0f to 0xcfe8155 2024-09-13T09:08:43.515 INFO:teuthology.orchestra.run.smithi031.stdout:4081 read 0x9d55a6f thru 0x9d573a5 (0x1937 bytes) 2024-09-13T09:08:43.517 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:43.517 INFO:teuthology.orchestra.run.smithi031.stdout:4083 writesame 0x1bdc7e4 thru 0x1be1fd3 data_size 0x70(0x57f0 bytes) 2024-09-13T09:08:43.520 INFO:teuthology.orchestra.run.smithi031.stdout:4084 read 0x5e19c19 thru 0x5e24e00 (0xb1e8 bytes) 2024-09-13T09:08:43.521 INFO:teuthology.orchestra.run.smithi031.stdout:4085 read 0xb0df2a9 thru 0xb0e461d (0x5375 bytes) 2024-09-13T09:08:43.522 INFO:teuthology.orchestra.run.smithi031.stdout:4086 punch from 0x37e2a7a to 0x37ec4e7, (0x9a6d bytes) 2024-09-13T09:08:43.522 INFO:teuthology.orchestra.run.smithi031.stdout:4087 write 0x4d291ad thru 0x4d30d7d (0x7bd1 bytes) 2024-09-13T09:08:43.524 INFO:teuthology.orchestra.run.smithi031.stdout:4088 write 0x9ff2273 thru 0x9ff4379 (0x2107 bytes) 2024-09-13T09:08:43.528 INFO:teuthology.orchestra.run.smithi031.stdout:4089 punch from 0x6fec96b to 0x6ff9798, (0xce2d bytes) 2024-09-13T09:08:43.528 INFO:teuthology.orchestra.run.smithi031.stdout:4090 read 0x4ab4b0 thru 0x4ad370 (0x1ec1 bytes) 2024-09-13T09:08:43.528 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:43.528 INFO:teuthology.orchestra.run.smithi031.stdout:4092 write 0x9cbe3a8 thru 0x9cc978c (0xb3e5 bytes) 2024-09-13T09:08:43.532 INFO:teuthology.orchestra.run.smithi031.stdout:4093 write 0xd3707a thru 0xd445bf (0xd546 bytes) 2024-09-13T09:08:43.536 INFO:teuthology.orchestra.run.smithi031.stdout:4097 write 0x51040eb thru 0x5110ef1 (0xce07 bytes) 2024-09-13T09:08:43.537 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:43.535+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:43.555 INFO:teuthology.orchestra.run.smithi031.stdout:4098 trunc from 0xcfe8155 to 0x6053bb 2024-09-13T09:08:43.580 INFO:teuthology.orchestra.run.smithi031.stdout:4099 read 0x2c5cc thru 0x34359 (0x7d8e bytes) 2024-09-13T09:08:43.582 INFO:teuthology.orchestra.run.smithi031.stdout:4100 punch from 0x4c51ab to 0x4d32bf, (0xe114 bytes) 2024-09-13T09:08:43.582 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:43.601 INFO:teuthology.orchestra.run.smithi031.stdout:4102 write 0x7cfc7d4 thru 0x7d041e5 (0x7a12 bytes) 2024-09-13T09:08:43.607 INFO:teuthology.orchestra.run.smithi031.stdout:4103 write 0x6fcbd35 thru 0x6fd4042 (0x830e bytes) 2024-09-13T09:08:43.611 INFO:teuthology.orchestra.run.smithi031.stdout:4104 read 0x413dcdb thru 0x414873f (0xaa65 bytes) 2024-09-13T09:08:43.633 INFO:teuthology.orchestra.run.smithi031.stdout:4105 writesame 0xed40e63 thru 0xed49bed data_size 0x1(0x8d8b bytes) 2024-09-13T09:08:43.633 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:43.633 INFO:teuthology.orchestra.run.smithi031.stdout:4108 read 0x20090b9 thru 0x200a503 (0x144b bytes) 2024-09-13T09:08:43.634 INFO:teuthology.orchestra.run.smithi031.stdout:4109 write 0xb8f9f1f thru 0xb9010c5 (0x71a7 bytes) 2024-09-13T09:08:43.635 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:43.635 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:43.638 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:43.633+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:43.641 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:43.641 INFO:teuthology.orchestra.run.smithi031.stdout:4111 write 0xd5f001b thru 0xd5f17f3 (0x17d9 bytes) 2024-09-13T09:08:43.648 INFO:teuthology.orchestra.run.smithi031.stdout:4112 trunc from 0xed49bee to 0xe7cdd60 2024-09-13T09:08:43.650 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:43.667 INFO:teuthology.orchestra.run.smithi031.stdout:4113 trunc from 0xe7cdd60 to 0x95ac76d 2024-09-13T09:08:43.685 INFO:teuthology.orchestra.run.smithi031.stdout:4114 write 0x719ec84 thru 0x71ac2cc (0xd649 bytes) 2024-09-13T09:08:43.688 INFO:teuthology.orchestra.run.smithi031.stdout:4115 clone 32 order 21 su 32768 sc 6 2024-09-13T09:08:43.739 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:43.733+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:43.791 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:43.792 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:43.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:43.847 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:43.847+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:43.941 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:43.937+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:43.952 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone31 from 0x95ac76d (overlap 0x6053bb) to 0x1ce5eb 2024-09-13T09:08:43.968 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:43.969 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:43.981 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:44.042 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:44.039+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:44.062 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-13T09:08:44.106 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:44.106 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:44.124 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:44.167 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:44.167+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:44.245 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:44.241+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:44.280 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:44.280 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:44.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:44.346 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:44.343+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:44.415 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:44.416 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:44.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:44.471 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:44.471+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:44.548 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:44.547+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:44.595 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:44.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:44.651 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:44.651+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:44.745 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:44.745 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:44.750 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:44.747+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:44.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:44.851 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:44.845+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:44.886 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:44.886 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:44.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:44.989 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:44.985+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:45.053 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:45.049+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:45.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:45.154 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:45.151+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:45.169 INFO:teuthology.orchestra.run.smithi031.stdout:4116 write 0x6f25875 thru 0x6f33429 (0xdbb5 bytes) 2024-09-13T09:08:45.246 INFO:teuthology.orchestra.run.smithi031.stdout:4117 write 0xaecb1e3 thru 0xaed99ed (0xe80b bytes) 2024-09-13T09:08:45.254 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:45.251+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:45.270 INFO:teuthology.orchestra.run.smithi031.stdout:4118 write 0xd8dd8f9 thru 0xd8e41e8 (0x68f0 bytes) 2024-09-13T09:08:45.276 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:45.276 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:45.285 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:08:45.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:45.300 INFO:teuthology.orchestra.run.smithi031.stdout:4120 trunc from 0xd8e41e9 to 0x376471d 2024-09-13T09:08:45.355 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:45.355+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:45.357 INFO:teuthology.orchestra.run.smithi031.stdout:4121 write 0xda64309 thru 0xda7129a (0xcf92 bytes) 2024-09-13T09:08:45.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:45.367 INFO:teuthology.orchestra.run.smithi031.stdout:4122 punch from 0x431539e to 0x4319c11, (0x4873 bytes) 2024-09-13T09:08:45.367 INFO:teuthology.orchestra.run.smithi031.stdout:4123 write 0x6eb299e thru 0x6eb2ebc (0x51f bytes) 2024-09-13T09:08:45.372 INFO:teuthology.orchestra.run.smithi031.stdout:4125 punch from 0x4000b1 to 0x409398, (0x92e7 bytes) 2024-09-13T09:08:45.372 INFO:teuthology.orchestra.run.smithi031.stdout:4126 write 0x5b5c593 thru 0x5b6b9cf (0xf43d bytes) 2024-09-13T09:08:45.383 INFO:teuthology.orchestra.run.smithi031.stdout:4127 writesame 0xb7f3fc3 thru 0xb8012e5 data_size 0x2b(0xd323 bytes) 2024-09-13T09:08:45.394 INFO:teuthology.orchestra.run.smithi031.stdout:4128 write 0xd09b54c thru 0xd09bccd (0x782 bytes) 2024-09-13T09:08:45.400 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:45.400 INFO:teuthology.orchestra.run.smithi031.stdout:4132 read 0xa5fd166 thru 0xa60ae98 (0xdd33 bytes) 2024-09-13T09:08:45.400 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:45.400 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:45.400 INFO:teuthology.orchestra.run.smithi031.stdout:4135 write 0x545c48f thru 0x545cdfa (0x96c bytes) 2024-09-13T09:08:45.404 INFO:teuthology.orchestra.run.smithi031.stdout:4136 write 0x235679f thru 0x2361365 (0xabc7 bytes) 2024-09-13T09:08:45.422 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:45.423 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:45.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:45.450 INFO:teuthology.orchestra.run.smithi031.stdout:4137 trunc from 0xda7129b to 0x4271006 2024-09-13T09:08:45.465 INFO:teuthology.orchestra.run.smithi031.stdout:4138 trunc from 0x4271006 to 0x374daa6 2024-09-13T09:08:45.475 INFO:teuthology.orchestra.run.smithi031.stdout:4140 read 0x1c2af8 thru 0x1d0197 (0xd6a0 bytes) 2024-09-13T09:08:45.477 INFO:teuthology.orchestra.run.smithi031.stdout:4142 read 0x41c54c thru 0x427448 (0xaefd bytes) 2024-09-13T09:08:45.478 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:45.485 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:45.481+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:45.490 INFO:teuthology.orchestra.run.smithi031.stdout:4145 write 0x7896823 thru 0x789d50c (0x6cea bytes) 2024-09-13T09:08:45.495 INFO:teuthology.orchestra.run.smithi031.stdout:4146 write 0x283e282 thru 0x283f4c6 (0x1245 bytes) 2024-09-13T09:08:45.514 INFO:teuthology.orchestra.run.smithi031.stdout:4148 trunc from 0x789d50d to 0x6eba309 2024-09-13T09:08:45.524 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:45.524 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:45.541 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:45.541 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:45.544 INFO:teuthology.orchestra.run.smithi031.stdout:4149 writesame 0xdc157de thru 0xdc21a79 data_size 0x6a(0xc29c bytes) 2024-09-13T09:08:45.544 INFO:teuthology.orchestra.run.smithi031.stdout:4151 read 0x30822a8 thru 0x308379e (0x14f7 bytes) 2024-09-13T09:08:45.545 INFO:teuthology.orchestra.run.smithi031.stdout:4152 write 0xb32ee thru 0xbbda4 (0x8ab7 bytes) 2024-09-13T09:08:45.557 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:45.553+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:45.559 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:45.559 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:45.567 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:45.567 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:45.575 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:45.575 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:45.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:45.594 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:45.595 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:45.615 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:45.615 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:45.633 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:45.634 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:45.652 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:45.652 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:45.677 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:45.673+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:45.759 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:45.753+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:45.759 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:45.759 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:45.759 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:45.759 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:45.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:45.790 INFO:teuthology.orchestra.run.smithi031.stdout:4153 read 0x22c18d6 thru 0x22d140b (0xfb36 bytes) 2024-09-13T09:08:45.798 INFO:teuthology.orchestra.run.smithi031.stdout:4154 trunc from 0xdc21a7a to 0xb1ca039 2024-09-13T09:08:45.812 INFO:teuthology.orchestra.run.smithi031.stdout:4156 writesame 0xffd7e thru 0x101aba data_size 0x5(0x1d3d bytes) 2024-09-13T09:08:45.859 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:45.859+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:45.869 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:08:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:4157 read 0x5bbc79e thru 0x5bc235d (0x5bc0 bytes) 2024-09-13T09:08:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:4159 punch from 0x2c1c3b3 to 0x2c1fb1e, (0x376b bytes) 2024-09-13T09:08:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:4160 read 0x606ff3e thru 0x6074111 (0x41d4 bytes) 2024-09-13T09:08:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:4162 writesame 0x3decc44 thru 0x3df0442 data_size 0x2f(0x37ff bytes) 2024-09-13T09:08:45.885 INFO:teuthology.orchestra.run.smithi031.stdout:4163 write 0xa1c7b3c thru 0xa1c8a3d (0xf02 bytes) 2024-09-13T09:08:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:4164 read 0x4be638e thru 0x4bf11bf (0xae32 bytes) 2024-09-13T09:08:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:4165 read 0x5219d8c thru 0x5222cd5 (0x8f4a bytes) 2024-09-13T09:08:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:4166 punch from 0x9bd20da to 0x9bdac8b, (0x8bb1 bytes) 2024-09-13T09:08:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:4171 punch from 0x7e51f1b to 0x7e5ef89, (0xd06e bytes) 2024-09-13T09:08:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:4172 writesame 0x98fdce5 thru 0x990833a data_size 0x5e(0xa656 bytes) 2024-09-13T09:08:45.907 INFO:teuthology.orchestra.run.smithi031.stdout:4173 writesame 0x243f0f0 thru 0x2446bc8 data_size 0xb(0x7ad9 bytes) 2024-09-13T09:08:45.959 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:45.960 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:45.960 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:45.960 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:45.960 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:45.960 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:45.953+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:45.966 INFO:teuthology.orchestra.run.smithi031.stdout:4174 writesame 0xe445b37 thru 0xe44e0db data_size 0x1(0x85a5 bytes) 2024-09-13T09:08:45.967 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:45.967 INFO:teuthology.orchestra.run.smithi031.stdout:4181 writesame 0x3ab7a63 thru 0x3ac63ac data_size 0xd(0xe94a bytes) 2024-09-13T09:08:45.973 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:45.973 INFO:teuthology.orchestra.run.smithi031.stdout:4183 punch from 0x19d9412 to 0x19de803, (0x53f1 bytes) 2024-09-13T09:08:45.973 INFO:teuthology.orchestra.run.smithi031.stdout:4184 writesame 0x31976f3 thru 0x31a55eb data_size 0x35(0xdef9 bytes) 2024-09-13T09:08:45.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:45.976 INFO:teuthology.orchestra.run.smithi031.stdout:4187 write 0x3d34f73 thru 0x3d392ab (0x4339 bytes) 2024-09-13T09:08:45.982 INFO:teuthology.orchestra.run.smithi031.stdout:4190 writesame 0xd6154c1 thru 0xd621bf7 data_size 0x1(0xc737 bytes) 2024-09-13T09:08:46.053 INFO:teuthology.orchestra.run.smithi031.stdout:4191 read 0xa330490 thru 0xa337e34 (0x79a5 bytes) 2024-09-13T09:08:46.055 INFO:teuthology.orchestra.run.smithi031.stdout:4192 writesame 0x1d52f03 thru 0x1d5f945 data_size 0xd(0xca43 bytes) 2024-09-13T09:08:46.061 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:46.057+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:46.063 INFO:teuthology.orchestra.run.smithi031.stdout:4194 read 0x1305ec5 thru 0x13103cd (0xa509 bytes) 2024-09-13T09:08:46.064 INFO:teuthology.orchestra.run.smithi031.stdout:4196 writesame 0x6a55d7 thru 0x6a7738 data_size 0x1(0x2162 bytes) 2024-09-13T09:08:46.078 INFO:teuthology.orchestra.run.smithi031.stdout:4198 punch from 0xfbd2fe to 0xfbe2ad, (0xfaf bytes) 2024-09-13T09:08:46.078 INFO:teuthology.orchestra.run.smithi031.stdout:4201 write 0xe1350c3 thru 0xe13a526 (0x5464 bytes) 2024-09-13T09:08:46.087 INFO:teuthology.orchestra.run.smithi031.stdout:4203 write 0xe00883a thru 0xe00b13c (0x2903 bytes) 2024-09-13T09:08:46.110 INFO:teuthology.orchestra.run.smithi031.stdout:4205 trunc from 0xe44e0dc to 0x89d7993 2024-09-13T09:08:46.152 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:46.152 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:46.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:46.152 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:46.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.152 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.152 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708645056999998, 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.153 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:46.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:08:46.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:08:46.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.157 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708620474, 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.160 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.164 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708596470000003, 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:08:46.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708572986999997, 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.169 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.173 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708549579999996, 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:08:46.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:08:46.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708526331000002, 2024-09-13T09:08:46.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708494224999995, 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.183 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:08:46.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708458608000001, 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:08:46.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708426627999998, 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.192 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:46.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.196 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708357348999996, 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:08:46.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708332288999998, 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.201 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:46.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708308858999999, 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:08:46.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:08:46.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708285476, 2024-09-13T09:08:46.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:08:46.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:08:46.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708262059999996, 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.215 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.219 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708238136999995, 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:08:46.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708205053, 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.224 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:46.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.228 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708181253000006, 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:08:46.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:08:46.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708156927000005, 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.233 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:08:46.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.237 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708124366999996, 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:46.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:08:46.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 70.708084494000005, 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.242 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:46.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:08:46.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:08:46.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.246 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.246 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.246 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:46.248 INFO:teuthology.orchestra.run.smithi031.stdout:4206 trunc from 0x89d7993 to 0x48bca91 2024-09-13T09:08:46.248 INFO:teuthology.orchestra.run.smithi031.stdout:4207 write 0x71b6b64 thru 0x71c5cd8 (0xf175 bytes) 2024-09-13T09:08:46.248 INFO:teuthology.orchestra.run.smithi031.stdout:4208 punch from 0x18b37ad to 0x18c0195, (0xc9e8 bytes) 2024-09-13T09:08:46.248 INFO:teuthology.orchestra.run.smithi031.stdout:4209 write 0xb972376 thru 0xb97d831 (0xb4bc bytes) 2024-09-13T09:08:46.248 INFO:teuthology.orchestra.run.smithi031.stdout:4210 writesame 0xa5c640d thru 0xa5c9b14 data_size 0xc(0x3708 bytes) 2024-09-13T09:08:46.248 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:46.247+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:46.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:46.256 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:08:46.256 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:08:46.256 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:08:46.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:08:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:4211 writesame 0xeb24d09 thru 0xeb29b15 data_size 0xd(0x4e0d bytes) 2024-09-13T09:08:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:4212 read 0x4f5dea5 thru 0x4f63fd6 (0x6132 bytes) 2024-09-13T09:08:46.264 INFO:teuthology.orchestra.run.smithi031.stdout:4214 write 0xae407d thru 0xae801c (0x3fa0 bytes) 2024-09-13T09:08:46.308 INFO:teuthology.orchestra.run.smithi031.stdout:4215 writesame 0xb2cb00a thru 0xb2d798d data_size 0x12(0xc984 bytes) 2024-09-13T09:08:46.318 INFO:teuthology.orchestra.run.smithi031.stdout:4216 read 0xa2f83d0 thru 0xa3080ef (0xfd20 bytes) 2024-09-13T09:08:46.319 INFO:teuthology.orchestra.run.smithi031.stdout:4217 read 0x6209bd7 thru 0x6218368 (0xe792 bytes) 2024-09-13T09:08:46.320 INFO:teuthology.orchestra.run.smithi031.stdout:4218 punch from 0x24761fb to 0x247f9b6, (0x97bb bytes) 2024-09-13T09:08:46.320 INFO:teuthology.orchestra.run.smithi031.stdout:4219 writesame 0xba6bfce thru 0xba70e80 data_size 0x1(0x4eb3 bytes) 2024-09-13T09:08:46.341 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:46.348 INFO:teuthology.orchestra.run.smithi031.stdout:4224 trunc from 0xeb29b16 to 0xb89352d 2024-09-13T09:08:46.348 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:46.345+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:46.360 INFO:teuthology.orchestra.run.smithi031.stdout:4225 read 0x3ea6ab thru 0x3f06ea (0x6040 bytes) 2024-09-13T09:08:46.361 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:46.361 INFO:teuthology.orchestra.run.smithi031.stdout:4228 read 0x732ed1f thru 0x733a4d5 (0xb7b7 bytes) 2024-09-13T09:08:46.365 INFO:teuthology.orchestra.run.smithi031.stdout:4229 trunc from 0xb89352d to 0xcd0bf7e 2024-09-13T09:08:46.374 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:46.374 INFO:teuthology.orchestra.run.smithi031.stdout:4233 read 0xaffc856 thru 0xb00b0af (0xe85a bytes) 2024-09-13T09:08:46.375 INFO:teuthology.orchestra.run.smithi031.stdout:4235 punch from 0x14ac871 to 0x14b6696, (0x9e25 bytes) 2024-09-13T09:08:46.375 INFO:teuthology.orchestra.run.smithi031.stdout:4236 read 0x76d490e thru 0x76dfea6 (0xb599 bytes) 2024-09-13T09:08:46.376 INFO:teuthology.orchestra.run.smithi031.stdout:4238 punch from 0xbf0e25b to 0xbf19f96, (0xbd3b bytes) 2024-09-13T09:08:46.376 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:46.376 INFO:teuthology.orchestra.run.smithi031.stdout:4240 read 0x849443a thru 0x84a222c (0xddf3 bytes) 2024-09-13T09:08:46.377 INFO:teuthology.orchestra.run.smithi031.stdout:4244 write 0x68afb38 thru 0x68bbeb1 (0xc37a bytes) 2024-09-13T09:08:46.388 INFO:teuthology.orchestra.run.smithi031.stdout:4245 punch from 0xb0541f6 to 0xb06002a, (0xbe34 bytes) 2024-09-13T09:08:46.389 INFO:teuthology.orchestra.run.smithi031.stdout:4247 punch from 0x605f11a to 0x6065d2a, (0x6c10 bytes) 2024-09-13T09:08:46.389 INFO:teuthology.orchestra.run.smithi031.stdout:4248 read 0x9f59b6 thru 0xa02c1c (0xd267 bytes) 2024-09-13T09:08:46.389 INFO:teuthology.orchestra.run.smithi031.stdout:4250 read 0x27f4c26 thru 0x27fc773 (0x7b4e bytes) 2024-09-13T09:08:46.394 INFO:teuthology.orchestra.run.smithi031.stdout:4251 trunc from 0xcd0bf7e to 0xebe99a3 2024-09-13T09:08:46.403 INFO:teuthology.orchestra.run.smithi031.stdout:4252 punch from 0x7b7b99c to 0x7b7ef79, (0x35dd bytes) 2024-09-13T09:08:46.409 INFO:teuthology.orchestra.run.smithi031.stdout:4253 trunc from 0xebe99a3 to 0xaa49e48 2024-09-13T09:08:46.420 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:46.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:46.421 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:46.421 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:46.426 INFO:teuthology.orchestra.run.smithi031.stdout:4254 write 0x2024c1f thru 0x202b3d3 (0x67b5 bytes) 2024-09-13T09:08:46.428 INFO:teuthology.orchestra.run.smithi031.stdout:4255 write 0x6646cf3 thru 0x664973f (0x2a4d bytes) 2024-09-13T09:08:46.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:46.442 INFO:teuthology.orchestra.run.smithi031.stdout:4257 trunc from 0xaa49e48 to 0x249e881 2024-09-13T09:08:46.481 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:46.477+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:46.519 INFO:teuthology.orchestra.run.smithi031.stdout:4258 read 0xe68095 thru 0xe77b19 (0xfa85 bytes) 2024-09-13T09:08:46.520 INFO:teuthology.orchestra.run.smithi031.stdout:4261 read 0x220efdd thru 0x221bd4d (0xcd71 bytes) 2024-09-13T09:08:46.521 INFO:teuthology.orchestra.run.smithi031.stdout:4262 read 0x3a2541 thru 0x3ad281 (0xad41 bytes) 2024-09-13T09:08:46.523 INFO:teuthology.orchestra.run.smithi031.stdout:4263 read 0x87e1e7 thru 0x888552 (0xa36c bytes) 2024-09-13T09:08:46.524 INFO:teuthology.orchestra.run.smithi031.stdout:4264 punch from 0x2070656 to 0x20767f3, (0x619d bytes) 2024-09-13T09:08:46.549 INFO:teuthology.orchestra.run.smithi031.stdout:4265 write 0x920646d thru 0x920a4d6 (0x406a bytes) 2024-09-13T09:08:46.550 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:46.545+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:46.556 INFO:teuthology.orchestra.run.smithi031.stdout:4266 read 0xd0391e thru 0xd059f1 (0x20d4 bytes) 2024-09-13T09:08:46.557 INFO:teuthology.orchestra.run.smithi031.stdout:4267 punch from 0x48a1d38 to 0x48aea72, (0xcd3a bytes) 2024-09-13T09:08:46.557 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:46.580 INFO:teuthology.orchestra.run.smithi031.stdout:4269 write 0xe6e01bc thru 0xe6e7fa8 (0x7ded bytes) 2024-09-13T09:08:46.587 INFO:teuthology.orchestra.run.smithi031.stdout:4270 punch from 0x46b7eb6 to 0x46c5929, (0xda73 bytes) 2024-09-13T09:08:46.587 INFO:teuthology.orchestra.run.smithi031.stdout:4272 punch from 0x38f7e56 to 0x38fba59, (0x3c03 bytes) 2024-09-13T09:08:46.587 INFO:teuthology.orchestra.run.smithi031.stdout:4273 writesame 0xb4d730d thru 0xb4e2414 data_size 0x6e(0xb108 bytes) 2024-09-13T09:08:46.595 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 95, 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T09:06:44.149006+0000", 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T09:08:45.859211+0000", 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T09:06:55.611487+0000", 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T09:07:49.811063+0000", 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T09:08:46.596 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-13T09:08:46.597 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:06:55.131330+0000", 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:46.598 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:46.599 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:46.600 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:46.601 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:46.602 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T09:08:46.603 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T09:08:46.604 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T09:08:46.605 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:02.743102+0000", 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:46.606 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:46.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.608 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-09-13T09:08:46.608 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:46.608 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:46.608 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:46.608 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:46.608 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:46.609 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T09:08:46.609 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-09-13T09:08:46.609 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:46.609 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:46.609 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:46.609 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:46.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:46.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:46.610 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:46.610 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:46.610 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:46.610 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:46.610 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:46.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:46.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:46.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:46.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:46.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:46.611 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:46.612 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:46.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:46.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.612 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:46.612 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:46.612 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:46.612 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:46.613 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:46.613 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:46.613 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:46.613 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:46.613 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:46.613 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:46.614 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:46.614 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:08:46.614 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:46.614 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:08:46.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.614 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:46.614 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 5, 2024-09-13T09:08:46.615 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 5, 2024-09-13T09:08:46.615 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.75, 2024-09-13T09:08:46.615 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 3.75, 2024-09-13T09:08:46.615 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 3.75, 2024-09-13T09:08:46.615 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:08:46.615 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:46.616 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:46.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.616 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:08:46.616 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T09:08:46.616 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:12.517158+0000", 2024-09-13T09:08:46.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:08:46.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:08:46.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:08:46.617 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:08:46.617 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:08:46.617 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:08:46.618 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:08:46.618 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:08:46.618 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:08:46.618 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:08:46.618 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:08:46.618 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:08:46.618 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T09:08:46.618 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T09:08:46.619 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T09:08:46.619 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T09:08:46.619 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T09:08:46.619 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:08:46.619 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:08:46.619 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:08:46.619 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:08:46.619 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:08:46.620 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:08:46.620 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:08:46.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.620 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "93", 2024-09-13T09:08:46.620 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:08:46.620 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:08:46.620 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:08:46.620 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:08:46.621 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:08:46.621 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 35, 2024-09-13T09:08:46.621 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 93, 2024-09-13T09:08:46.621 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:08:46.621 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:08:46.621 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:08:46.621 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:08:46.622 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:08:46.623 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:08:46.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.625 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "33c6fc7d-9828-4500-b103-b96d1cd7f420", 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.247772216796875, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 93, 2024-09-13T09:08:46.626 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:46.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6818/3291756747", 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6822/3291756747", 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6830/3291756747", 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6826/3291756747", 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:46.631 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "869ea3c5-8e6e-4b49-b70e-083f242a7aae", 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.999481201171875, 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:46.632 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 92, 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:46.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/533520566", 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/533520566", 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/533520566", 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/533520566", 2024-09-13T09:08:46.637 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e144d496-76ee-46e0-bdc6-135c77128535", 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:46.638 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:46.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.640 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.642 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.643 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6819/2253604482", 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6823/2253604482", 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/2253604482", 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6827/2253604482", 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:46.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "696b9414-9c19-4984-a8ff-14548e46b357", 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:46.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/3367214693", 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/3367214693", 2024-09-13T09:08:46.650 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/3367214693", 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/3367214693", 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ac58040d-9b54-42ca-b143-1816abb54816", 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:46.651 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6800", 2024-09-13T09:08:46.652 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6801", 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6802", 2024-09-13T09:08:46.654 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:46.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.654 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6803", 2024-09-13T09:08:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6806", 2024-09-13T09:08:46.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:46.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.656 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6807", 2024-09-13T09:08:46.656 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:46.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.657 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:46.657 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.658 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6804", 2024-09-13T09:08:46.658 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:46.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.659 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6805", 2024-09-13T09:08:46.659 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:08:46.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.659 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6801/3778074750", 2024-09-13T09:08:46.659 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6803/3778074750", 2024-09-13T09:08:46.660 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6807/3778074750", 2024-09-13T09:08:46.660 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6805/3778074750", 2024-09-13T09:08:46.660 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:46.660 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:46.660 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:46.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.661 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:08:46.661 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2", 2024-09-13T09:08:46.661 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:46.661 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:46.661 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:46.661 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:46.662 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:46.662 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:46.662 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T09:08:46.662 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 52, 2024-09-13T09:08:46.662 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:46.662 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:46.663 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:46.663 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.664 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6808", 2024-09-13T09:08:46.664 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:46.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.664 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6809", 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6811", 2024-09-13T09:08:46.665 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6813", 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6819", 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6824", 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6815", 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6817", 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6809/1926603087", 2024-09-13T09:08:46.668 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6813/1926603087", 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6824/1926603087", 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6817/1926603087", 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d", 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6820", 2024-09-13T09:08:46.670 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6821", 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.671 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6823", 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6825", 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.672 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6830", 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6831", 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.673 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6827", 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6828", 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.674 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6821/766305500", 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6825/766305500", 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6831/766305500", 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6828/766305500", 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f36c332c-9497-47e8-b943-f6b6dcaebeb8", 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:08:46.675 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6810", 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6812", 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6814", 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6816", 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6826", 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6829", 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6818", 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6822", 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6812/1032956039", 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6816/1032956039", 2024-09-13T09:08:46.680 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6829/1032956039", 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6822/1032956039", 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:46.681 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.183436+0000", 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:08:46.682 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:46.683 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:46.683 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:46.683 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:46.683 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:46.683 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.264080+0000", 2024-09-13T09:08:46.683 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:46.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.253587+0000", 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:08:46.684 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.244429+0000", 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:46.685 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.369458+0000", 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.158089+0000", 2024-09-13T09:08:46.686 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.217295+0000", 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.332254+0000", 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-13T09:08:46.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.a", 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: 4, 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: 2 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [ 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.f", 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:46.689 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 1, 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 0 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T09:08:46.690 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T09:08:46.691 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T09:08:46.692 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T09:08:46.692 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T09:08:46.692 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T09:08:46.692 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:08:46.692 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:08:46.692 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T09:08:46.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:46.692 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:46.693 INFO:teuthology.orchestra.run.smithi031.stdout:4276 write 0x1559c8c thru 0x1564ce3 (0xb058 bytes) 2024-09-13T09:08:46.694 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:46.694 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:46.695 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:46.695 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:46.695 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:46.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:46.696 INFO:tasks.thrashosds.thrasher:j is {'epoch': 95, 'fsid': 'fe16bf6d-4166-4417-80fd-1123617a33d3', 'created': '2024-09-13T09:06:44.149006+0000', 'modified': '2024-09-13T09:08:45.859211+0000', 'last_up_change': '2024-09-13T09:06:55.611487+0000', 'last_in_change': '2024-09-13T09:07:49.811063+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:06:55.131330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:07:02.743102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 5, 'score_stable': 5, 'optimal_score': 0.75, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:07:12.517158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 93, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '33c6fc7d-9828-4500-b103-b96d1cd7f420', 'up': 1, 'in': 1, 'weight': 0.247772216796875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6818', 'nonce': 3291756747}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6822', 'nonce': 3291756747}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6830', 'nonce': 3291756747}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6826', 'nonce': 3291756747}]}, 'public_addr': '172.21.15.31:6818/3291756747', 'cluster_addr': '172.21.15.31:6822/3291756747', 'heartbeat_back_addr': '172.21.15.31:6830/3291756747', 'heartbeat_front_addr': '172.21.15.31:6826/3291756747', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.999481201171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 533520566}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 533520566}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 533520566}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 533520566}]}, 'public_addr': '172.21.15.31:6801/533520566', 'cluster_addr': '172.21.15.31:6803/533520566', 'heartbeat_back_addr': '172.21.15.31:6807/533520566', 'heartbeat_front_addr': '172.21.15.31:6805/533520566', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e144d496-76ee-46e0-bdc6-135c77128535', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6817', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2253604482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6821', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2253604482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6829', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 2253604482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6825', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 2253604482}]}, 'public_addr': '172.21.15.31:6819/2253604482', 'cluster_addr': '172.21.15.31:6823/2253604482', 'heartbeat_back_addr': '172.21.15.31:6831/2253604482', 'heartbeat_front_addr': '172.21.15.31:6827/2253604482', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '696b9414-9c19-4984-a8ff-14548e46b357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 3367214693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 3367214693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 3367214693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 3367214693}]}, 'public_addr': '172.21.15.31:6809/3367214693', 'cluster_addr': '172.21.15.31:6811/3367214693', 'heartbeat_back_addr': '172.21.15.31:6815/3367214693', 'heartbeat_front_addr': '172.21.15.31:6813/3367214693', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ac58040d-9b54-42ca-b143-1816abb54816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6800', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6801', 'nonce': 3778074750}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 3778074750}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 3778074750}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 3778074750}]}, 'public_addr': '172.21.15.60:6801/3778074750', 'cluster_addr': '172.21.15.60:6803/3778074750', 'heartbeat_back_addr': '172.21.15.60:6807/3778074750', 'heartbeat_front_addr': '172.21.15.60:6805/3778074750', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1926603087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6811', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 1926603087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6819', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6824', 'nonce': 1926603087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6815', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 1926603087}]}, 'public_addr': '172.21.15.60:6809/1926603087', 'cluster_addr': '172.21.15.60:6813/1926603087', 'heartbeat_back_addr': '172.21.15.60:6824/1926603087', 'heartbeat_front_addr': '172.21.15.60:6817/1926603087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 766305500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6823', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 766305500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 766305500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6827', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6828', 'nonce': 766305500}]}, 'public_addr': '172.21.15.60:6821/766305500', 'cluster_addr': '172.21.15.60:6825/766305500', 'heartbeat_back_addr': '172.21.15.60:6831/766305500', 'heartbeat_front_addr': '172.21.15.60:6828/766305500', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f36c332c-9497-47e8-b943-f6b6dcaebeb8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6812', 'nonce': 1032956039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6816', 'nonce': 1032956039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 1032956039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6822', 'nonce': 1032956039}]}, 'public_addr': '172.21.15.60:6812/1032956039', 'cluster_addr': '172.21.15.60:6816/1032956039', 'heartbeat_back_addr': '172.21.15.60:6829/1032956039', 'heartbeat_front_addr': '172.21.15.60:6822/1032956039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.183436+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.264080+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.253587+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.244429+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.369458+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.158089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.217295+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.332254+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.a', 'osds': [4, 2]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 1, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:08:46.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4277 punch from 0xb03383c to 0xb03e3ad, (0xab71 bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4278 read 0x2d637af thru 0x2d693a1 (0x5bf3 bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4279 writesame 0xd4c2874 thru 0xd4cf2bb data_size 0x4(0xca48 bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4280 write 0x2eb7900 thru 0x2eb9229 (0x192a bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4282 write 0x1021524 thru 0x102568b (0x4168 bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4284 trunc from 0xe6e7fa9 to 0xae97d04 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4285 read 0x6063f96 thru 0x6064710 (0x77b bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4287 write 0xd2bd6e9 thru 0xd2c2995 (0x52ad bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4289 write 0xd4aaa0e thru 0xd4b72cc (0xc8bf bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:4290 punch from 0x87d7763 to 0x87df645, (0x7ee2 bytes) 2024-09-13T09:08:46.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:46.699 INFO:teuthology.orchestra.run.smithi031.stdout:4292 writesame 0xb69c5f thru 0xb747e4 data_size 0x5(0xab86 bytes) 2024-09-13T09:08:46.699 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:46.691+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:46.699 INFO:teuthology.orchestra.run.smithi031.stdout:4293 read 0xee5697 thru 0xeeb4dc (0x5e46 bytes) 2024-09-13T09:08:46.699 INFO:teuthology.orchestra.run.smithi031.stdout:4294 write 0xb9cf418 thru 0xb9dd2a9 (0xde92 bytes) 2024-09-13T09:08:46.708 INFO:teuthology.orchestra.run.smithi031.stdout:4295 writesame 0xde6a24 thru 0xdf4c53 data_size 0x58(0xe230 bytes) 2024-09-13T09:08:46.729 INFO:teuthology.orchestra.run.smithi031.stdout:4296 read 0xc736fab thru 0xc73e8f6 (0x794c bytes) 2024-09-13T09:08:46.729 INFO:teuthology.orchestra.run.smithi031.stdout:4297 write 0x831c03c thru 0x832283c (0x6801 bytes) 2024-09-13T09:08:46.763 INFO:teuthology.orchestra.run.smithi031.stdout:4300 trunc from 0xd4b72cd to 0xa639e4 2024-09-13T09:08:46.794 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:46.791+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:46.809 INFO:teuthology.orchestra.run.smithi031.stdout:4301 read 0x5b4f2 thru 0x62ac4 (0x75d3 bytes) 2024-09-13T09:08:46.810 INFO:teuthology.orchestra.run.smithi031.stdout:4302 read 0x9539b0 thru 0x961c37 (0xe288 bytes) 2024-09-13T09:08:46.811 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:46.811 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:46.831 INFO:teuthology.orchestra.run.smithi031.stdout:4305 trunc from 0xa639e4 to 0xc95a507 2024-09-13T09:08:46.840 INFO:teuthology.orchestra.run.smithi031.stdout:4306 write 0x695626 thru 0x6a4c4e (0xf629 bytes) 2024-09-13T09:08:46.845 INFO:teuthology.orchestra.run.smithi031.stdout:4307 read 0x6980e2b thru 0x6981638 (0x80e bytes) 2024-09-13T09:08:46.845 INFO:teuthology.orchestra.run.smithi031.stdout:4308 read 0x2d1972c thru 0x2d26a6c (0xd341 bytes) 2024-09-13T09:08:46.845 INFO:teuthology.orchestra.run.smithi031.stdout:4309 punch from 0x8fdf5d6 to 0x8fe1b89, (0x25b3 bytes) 2024-09-13T09:08:46.845 INFO:teuthology.orchestra.run.smithi031.stdout:4310 write 0x36e2a96 thru 0x36e4382 (0x18ed bytes) 2024-09-13T09:08:46.864 INFO:teuthology.orchestra.run.smithi031.stdout:4312 trunc from 0xc95a507 to 0x44d2400 2024-09-13T09:08:46.878 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:46.878 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653635179999995, 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:08:46.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.880 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:46.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:08:46.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653604329999993, 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:08:46.884 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:08:46.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:08:46.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653578163000006, 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:08:46.889 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:08:46.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:08:46.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653538689000001, 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:08:46.894 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:08:46.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653504135000006, 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:08:46.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:08:46.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:08:46.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:08:46.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:08:46.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:08:46.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.905 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653471779, 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.906 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:08:46.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.910 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653439508000005, 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:08:46.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:08:46.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:08:46.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653405472000003, 2024-09-13T09:08:46.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:08:46.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:08:46.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:08:46.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:08:46.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:08:46.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:08:46.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.922 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653372207999993, 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:46.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:08:46.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:08:46.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.653339967999997, 2024-09-13T09:08:46.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:46.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:46.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:08:46.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:08:46.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:46.934 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.644359882000003, 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:46.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:08:46.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.644319002000003, 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:08:46.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.644278733999997, 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.944 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:08:46.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.644231691000002, 2024-09-13T09:08:46.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.644186129000005, 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:08:46.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:08:46.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.644095961000005, 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:08:46.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:08:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:08:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:08:46.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.644049406999997, 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:08:46.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:08:46.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:08:46.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.642408482999997, 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.968 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:46.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:08:46.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.642342677000002, 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.973 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:08:46.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:08:46.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:08:46.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.978 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.979 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:08:46.979 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:46.979 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 13.959335032, 2024-09-13T09:08:46.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:08:46.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:46.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:46.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:46.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:46.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:46.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:08:46.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.980 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:46.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:46.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:46.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:46.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:46.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:46.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:46.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:08:46.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:46.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:46.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:08:46.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:08:46.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:46.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:08:46.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:08:46.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:46.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:08:46.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:08:46.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:46.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:08:46.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:08:46.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:46.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:08:46.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:08:46.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:08:46.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:08:46.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:08:46.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:08:46.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:08:46.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:08:46.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:46.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:08:46.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:08:46.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:08:46.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:08:46.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:08:46.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:46.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:46.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:46.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:08:46.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:08:46.992 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.992 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.992 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:46.993 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:46.993 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:46.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:47.000 INFO:teuthology.orchestra.run.smithi031.stdout:4313 write 0x589d505 thru 0x58a585e (0x835a bytes) 2024-09-13T09:08:47.000 INFO:teuthology.orchestra.run.smithi031.stdout:4314 trunc from 0x58a585f to 0xda52cf0 2024-09-13T09:08:47.000 INFO:teuthology.orchestra.run.smithi031.stdout:4315 read 0xec0a2c thru 0xec1ce8 (0x12bd bytes) 2024-09-13T09:08:47.000 INFO:teuthology.orchestra.run.smithi031.stdout:4316 write 0x1edb35e thru 0x1ee8a82 (0xd725 bytes) 2024-09-13T09:08:47.000 INFO:teuthology.orchestra.run.smithi031.stdout:4318 read 0xa4de47e thru 0xa4df2e2 (0xe65 bytes) 2024-09-13T09:08:47.000 INFO:teuthology.orchestra.run.smithi031.stdout:4319 read 0x50c0656 thru 0x50c608c (0x5a37 bytes) 2024-09-13T09:08:47.001 INFO:teuthology.orchestra.run.smithi031.stdout:4320 write 0x935cea9 thru 0x9360dfc (0x3f54 bytes) 2024-09-13T09:08:47.001 INFO:teuthology.orchestra.run.smithi031.stdout:4321 write 0x76f4942 thru 0x76fe591 (0x9c50 bytes) 2024-09-13T09:08:47.001 INFO:teuthology.orchestra.run.smithi031.stdout:4322 trunc from 0xda52cf0 to 0x8bd7d3d 2024-09-13T09:08:47.001 INFO:teuthology.orchestra.run.smithi031.stdout:4324 write 0xca6228 thru 0xcb4b60 (0xe939 bytes) 2024-09-13T09:08:47.001 INFO:teuthology.orchestra.run.smithi031.stdout:4326 write 0x93ee416 thru 0x93fc7f0 (0xe3db bytes) 2024-09-13T09:08:47.002 INFO:teuthology.orchestra.run.smithi031.stdout:4327 read 0x402c3af thru 0x4038dc5 (0xca17 bytes) 2024-09-13T09:08:47.002 INFO:teuthology.orchestra.run.smithi031.stdout:4328 read 0x7bb9028 thru 0x7bb94d0 (0x4a9 bytes) 2024-09-13T09:08:47.002 INFO:teuthology.orchestra.run.smithi031.stdout:4329 write 0xf83bf1 thru 0xf8786f (0x3c7f bytes) 2024-09-13T09:08:47.002 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:47.002 INFO:teuthology.orchestra.run.smithi031.stdout:4332 punch from 0x4f785fc to 0x4f87a94, (0xf498 bytes) 2024-09-13T09:08:47.003 INFO:teuthology.orchestra.run.smithi031.stdout:4333 read 0x434421c thru 0x434cd2d (0x8b12 bytes) 2024-09-13T09:08:47.003 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:47.003 INFO:teuthology.orchestra.run.smithi031.stdout:4339 punch from 0x11d8130 to 0x11e126a, (0x913a bytes) 2024-09-13T09:08:47.003 INFO:teuthology.orchestra.run.smithi031.stdout:4340 writesame 0x60bbaad thru 0x60bdc31 data_size 0x1(0x2185 bytes) 2024-09-13T09:08:47.004 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:46.991+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:47.006 INFO:teuthology.orchestra.run.smithi031.stdout:4341 read 0x518fc33 thru 0x5195843 (0x5c11 bytes) 2024-09-13T09:08:47.006 INFO:teuthology.orchestra.run.smithi031.stdout:4342 write 0xcbb354 thru 0xcbdb9b (0x2848 bytes) 2024-09-13T09:08:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:4344 punch from 0x12d8028 to 0x12e10b8, (0x9090 bytes) 2024-09-13T09:08:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:4346 read 0x6943661 thru 0x694a306 (0x6ca6 bytes) 2024-09-13T09:08:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:4348 read 0x90f6708 thru 0x910122d (0xab26 bytes) 2024-09-13T09:08:47.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:47.014 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:47.014 INFO:teuthology.orchestra.run.smithi031.stdout:4352 clone 33 order 19 su 16384 sc 15 2024-09-13T09:08:47.066 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:08:47.067 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:08:47.086 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-13T09:08:45.862681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1215526861,"num_objects":949,"num_object_clones":206,"num_object_copies":1898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":949,"num_whiteouts":255,"num_read":148039,"num_read_kb":487660,"num_write":14923,"num_write_kb":266460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":903,"num_bytes_recovered":120824171,"num_keys_recovered":268,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11764,"ondisk_log_size":11764,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":611120,"kb_used_data":303180,"kb_used_omap":12,"kb_used_meta":307827,"kb_avail":754363600,"statfs":{"total":773094113280,"available":772468326400,"internally_reserved":0,"allocated":310456320,"data_stored":325979495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12744,"internal_metadata":315215416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":77468979,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":824,"num_read_kb":11827,"num_write":267,"num_write_kb":8022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.620656"},"pg_stats":[{"pgid":"3.1f","version":"89'379","reported_seq":3382,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:08:41.225079+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:08:41.225079+0000","last_peered":"2024-09-13T09:08:41.225079+0000","last_clean":"2024-09-13T09:08:41.225079+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:08:41.225079+0000","last_undegraded":"2024-09-13T09:08:41.225079+0000","last_fullsized":"2024-09-13T09:08:41.225079+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":30293929,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":4015,"num_read_kb":12669,"num_write":530,"num_write_kb":3842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"89'431","reported_seq":2369,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:08:41.225144+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:08:41.225144+0000","last_peered":"2024-09-13T09:08:41.225144+0000","last_clean":"2024-09-13T09:08:41.225144+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:08:41.225144+0000","last_undegraded":"2024-09-13T09:08:41.225144+0000","last_fullsized":"2024-09-13T09:08:41.225144+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":42408982,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":3716,"num_read_kb":10925,"num_write":512,"num_write_kb":4440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"89'1227","reported_seq":4262,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.412086+0000","last_change":"2024-09-13T09:08:38.210944+0000","last_active":"2024-09-13T09:08:40.412086+0000","last_peered":"2024-09-13T09:08:40.412086+0000","last_clean":"2024-09-13T09:08:40.412086+0000","last_became_active":"2024-09-13T09:08:38.208191+0000","last_became_peered":"2024-09-13T09:08:38.208191+0000","last_unstale":"2024-09-13T09:08:40.412086+0000","last_undegraded":"2024-09-13T09:08:40.412086+0000","last_fullsized":"2024-09-13T09:08:40.412086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:04.686300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":51068517,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":11,"num_read":7990,"num_read_kb":30849,"num_write":2103,"num_write_kb":15622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"89'343","reported_seq":2602,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.349319+0000","last_change":"2024-09-13T09:08:38.212078+0000","last_active":"2024-09-13T09:08:40.349319+0000","last_peered":"2024-09-13T09:08:40.349319+0000","last_clean":"2024-09-13T09:08:40.349319+0000","last_became_active":"2024-09-13T09:08:38.209451+0000","last_became_peered":"2024-09-13T09:08:38.209451+0000","last_unstale":"2024-09-13T09:08:40.349319+0000","last_undegraded":"2024-09-13T09:08:40.349319+0000","last_fullsized":"2024-09-13T09:08:40.349319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:59.672566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":67062001,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":7,"num_read":4050,"num_read_kb":25721,"num_write":429,"num_write_kb":11221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"93'435","reported_seq":20355,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T09:08:44.142834+0000","last_change":"2024-09-13T09:08:38.212581+0000","last_active":"2024-09-13T09:08:44.142834+0000","last_peered":"2024-09-13T09:08:44.142834+0000","last_clean":"2024-09-13T09:08:44.142834+0000","last_became_active":"2024-09-13T09:08:38.210333+0000","last_became_peered":"2024-09-13T09:08:38.210333+0000","last_unstale":"2024-09-13T09:08:44.142834+0000","last_undegraded":"2024-09-13T09:08:44.142834+0000","last_fullsized":"2024-09-13T09:08:44.142834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:53.493070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":84728717,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":10,"num_read":21719,"num_read_kb":40724,"num_write":505,"num_write_kb":18010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"89'390","reported_seq":2958,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.377173+0000","last_change":"2024-09-13T09:08:09.864519+0000","last_active":"2024-09-13T09:08:40.377173+0000","last_peered":"2024-09-13T09:08:40.377173+0000","last_clean":"2024-09-13T09:08:40.377173+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:08:40.377173+0000","last_undegraded":"2024-09-13T09:08:40.377173+0000","last_fullsized":"2024-09-13T09:08:40.377173+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":47006373,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":4327,"num_read_kb":14969,"num_write":556,"num_write_kb":17106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"93'200","reported_seq":948,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T09:08:44.143254+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:08:44.143254+0000","last_peered":"2024-09-13T09:08:44.143254+0000","last_clean":"2024-09-13T09:08:44.143254+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:08:44.143254+0000","last_undegraded":"2024-09-13T09:08:44.143254+0000","last_fullsized":"2024-09-13T09:08:44.143254+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":12039194,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":7,"num_read":659,"num_read_kb":5032,"num_write":197,"num_write_kb":6294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"89'431","reported_seq":1924,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:08:41.505860+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:08:41.505860+0000","last_peered":"2024-09-13T09:08:41.505860+0000","last_clean":"2024-09-13T09:08:41.505860+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:08:41.505860+0000","last_undegraded":"2024-09-13T09:08:41.505860+0000","last_fullsized":"2024-09-13T09:08:41.505860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":40824373,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":7,"num_read":2855,"num_read_kb":10343,"num_write":519,"num_write_kb":6792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"89'346","reported_seq":4058,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:08:41.225011+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:08:41.225011+0000","last_peered":"2024-09-13T09:08:41.225011+0000","last_clean":"2024-09-13T09:08:41.225011+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:08:41.225011+0000","last_undegraded":"2024-09-13T09:08:41.225011+0000","last_fullsized":"2024-09-13T09:08:41.225011+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":47854503,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":4698,"num_read_kb":25739,"num_write":370,"num_write_kb":16666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"89'502","reported_seq":1091,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.460368+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:08:40.460368+0000","last_peered":"2024-09-13T09:08:40.460368+0000","last_clean":"2024-09-13T09:08:40.460368+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:08:40.460368+0000","last_undegraded":"2024-09-13T09:08:40.460368+0000","last_fullsized":"2024-09-13T09:08:40.460368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":28878004,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":2322,"num_read_kb":9752,"num_write":768,"num_write_kb":6449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"93'400","reported_seq":3028,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T09:08:45.192567+0000","last_change":"2024-09-13T09:08:38.222834+0000","last_active":"2024-09-13T09:08:45.192567+0000","last_peered":"2024-09-13T09:08:45.192567+0000","last_clean":"2024-09-13T09:08:45.192567+0000","last_became_active":"2024-09-13T09:08:38.221484+0000","last_became_peered":"2024-09-13T09:08:38.221484+0000","last_unstale":"2024-09-13T09:08:45.192567+0000","last_undegraded":"2024-09-13T09:08:45.192567+0000","last_fullsized":"2024-09-13T09:08:45.192567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:04.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":90672865,"num_objects":41,"num_object_clones":13,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":3514,"num_read_kb":21752,"num_write":397,"num_write_kb":13199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"89'438","reported_seq":17590,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.271431+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:08:40.271431+0000","last_peered":"2024-09-13T09:08:40.271431+0000","last_clean":"2024-09-13T09:08:40.271431+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:08:40.271431+0000","last_undegraded":"2024-09-13T09:08:40.271431+0000","last_fullsized":"2024-09-13T09:08:40.271431+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":28372414,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":7,"num_read":18936,"num_read_kb":29708,"num_write":408,"num_write_kb":9096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T09:08:39.213115+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:08:39.213115+0000","last_peered":"2024-09-13T09:08:39.213115+0000","last_clean":"2024-09-13T09:08:39.213115+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:08:39.213115+0000","last_undegraded":"2024-09-13T09:08:39.213115+0000","last_fullsized":"2024-09-13T09:08:39.213115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"94'474","reported_seq":3486,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T09:08:45.429044+0000","last_change":"2024-09-13T09:07:16.108420+0000","last_active":"2024-09-13T09:08:45.429044+0000","last_peered":"2024-09-13T09:08:45.429044+0000","last_clean":"2024-09-13T09:08:45.429044+0000","last_became_active":"2024-09-13T09:07:13.857484+0000","last_became_peered":"2024-09-13T09:07:13.857484+0000","last_unstale":"2024-09-13T09:08:45.429044+0000","last_undegraded":"2024-09-13T09:08:45.429044+0000","last_fullsized":"2024-09-13T09:08:45.429044+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:16:20.641628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":37453336,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":4406,"num_read_kb":9640,"num_write":644,"num_write_kb":6168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T09:08:43.864645+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:08:43.864645+0000","last_peered":"2024-09-13T09:08:43.864645+0000","last_clean":"2024-09-13T09:08:43.864645+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:08:43.864645+0000","last_undegraded":"2024-09-13T09:08:43.864645+0000","last_fullsized":"2024-09-13T09:08:43.864645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"90'273","reported_seq":1691,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T09:08:44.134645+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:08:44.134645+0000","last_peered":"2024-09-13T09:08:44.134645+0000","last_clean":"2024-09-13T09:08:44.134645+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:08:44.134645+0000","last_undegraded":"2024-09-13T09:08:44.134645+0000","last_fullsized":"2024-09-13T09:08:44.134645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":23703824,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":2182,"num_read_kb":13245,"num_write":270,"num_write_kb":4902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T09:08:39.213672+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:08:39.213672+0000","last_peered":"2024-09-13T09:08:39.213672+0000","last_clean":"2024-09-13T09:08:39.213672+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:08:39.213672+0000","last_undegraded":"2024-09-13T09:08:39.213672+0000","last_fullsized":"2024-09-13T09:08:39.213672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'341","reported_seq":3432,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T09:08:44.136221+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:08:44.136221+0000","last_peered":"2024-09-13T09:08:44.136221+0000","last_clean":"2024-09-13T09:08:44.136221+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:08:44.136221+0000","last_undegraded":"2024-09-13T09:08:44.136221+0000","last_fullsized":"2024-09-13T09:08:44.136221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":341,"log_dups_size":0,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":37150446,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":3994,"num_read_kb":16053,"num_write":500,"num_write_kb":3798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":86,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:08:41.224963+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:08:41.224963+0000","last_peered":"2024-09-13T09:08:41.224963+0000","last_clean":"2024-09-13T09:08:41.224963+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:08:41.224963+0000","last_undegraded":"2024-09-13T09:08:41.224963+0000","last_fullsized":"2024-09-13T09:08:41.224963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"89'336","reported_seq":2363,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.404864+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:08:40.404864+0000","last_peered":"2024-09-13T09:08:40.404864+0000","last_clean":"2024-09-13T09:08:40.404864+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:08:40.404864+0000","last_undegraded":"2024-09-13T09:08:40.404864+0000","last_fullsized":"2024-09-13T09:08:40.404864+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":27499376,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":5,"num_read":2926,"num_read_kb":14483,"num_write":471,"num_write_kb":9678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"89'242","reported_seq":1435,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.349480+0000","last_change":"2024-09-13T09:08:38.211675+0000","last_active":"2024-09-13T09:08:40.349480+0000","last_peered":"2024-09-13T09:08:40.349480+0000","last_clean":"2024-09-13T09:08:40.349480+0000","last_became_active":"2024-09-13T09:08:38.209914+0000","last_became_peered":"2024-09-13T09:08:38.209914+0000","last_unstale":"2024-09-13T09:08:40.349480+0000","last_undegraded":"2024-09-13T09:08:40.349480+0000","last_fullsized":"2024-09-13T09:08:40.349480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:19.947594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":43535580,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":1639,"num_read_kb":7611,"num_write":255,"num_write_kb":2286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":918446,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T09:08:39.213200+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:08:39.213200+0000","last_peered":"2024-09-13T09:08:39.213200+0000","last_clean":"2024-09-13T09:08:39.213200+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:08:39.213200+0000","last_undegraded":"2024-09-13T09:08:39.213200+0000","last_fullsized":"2024-09-13T09:08:39.213200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"89'216","reported_seq":1722,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-13T09:08:41.224702+0000","last_change":"2024-09-13T09:07:15.854106+0000","last_active":"2024-09-13T09:08:41.224702+0000","last_peered":"2024-09-13T09:08:41.224702+0000","last_clean":"2024-09-13T09:08:41.224702+0000","last_became_active":"2024-09-13T09:07:13.862285+0000","last_became_peered":"2024-09-13T09:07:13.862285+0000","last_unstale":"2024-09-13T09:08:41.224702+0000","last_undegraded":"2024-09-13T09:08:41.224702+0000","last_fullsized":"2024-09-13T09:08:41.224702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:44.617629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":14054869,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":1507,"num_read_kb":10479,"num_write":214,"num_write_kb":5403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":83,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T09:08:39.373864+0000","last_change":"2024-09-13T09:07:06.906706+0000","last_active":"2024-09-13T09:08:39.373864+0000","last_peered":"2024-09-13T09:08:39.373864+0000","last_clean":"2024-09-13T09:08:39.373864+0000","last_became_active":"2024-09-13T09:07:04.132570+0000","last_became_peered":"2024-09-13T09:07:04.132570+0000","last_unstale":"2024-09-13T09:08:39.373864+0000","last_undegraded":"2024-09-13T09:08:39.373864+0000","last_fullsized":"2024-09-13T09:08:39.373864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:14:50.380875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061195800000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"89'261","reported_seq":1283,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T09:08:43.958264+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:08:43.958264+0000","last_peered":"2024-09-13T09:08:43.958264+0000","last_clean":"2024-09-13T09:08:43.958264+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:08:43.958264+0000","last_undegraded":"2024-09-13T09:08:43.958264+0000","last_fullsized":"2024-09-13T09:08:43.958264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":12477064,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":1323,"num_read_kb":11001,"num_write":258,"num_write_kb":5932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":84,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T09:08:39.213000+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:08:39.213000+0000","last_peered":"2024-09-13T09:08:39.213000+0000","last_clean":"2024-09-13T09:08:39.213000+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:08:39.213000+0000","last_undegraded":"2024-09-13T09:08:39.213000+0000","last_fullsized":"2024-09-13T09:08:39.213000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"89'274","reported_seq":640,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T09:08:44.143904+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:08:44.143904+0000","last_peered":"2024-09-13T09:08:44.143904+0000","last_clean":"2024-09-13T09:08:44.143904+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:08:44.143904+0000","last_undegraded":"2024-09-13T09:08:44.143904+0000","last_fullsized":"2024-09-13T09:08:44.143904+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":7106227,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":7,"num_read":900,"num_read_kb":4527,"num_write":311,"num_write_kb":2479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":85,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T09:08:39.213336+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:08:39.213336+0000","last_peered":"2024-09-13T09:08:39.213336+0000","last_clean":"2024-09-13T09:08:39.213336+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:08:39.213336+0000","last_undegraded":"2024-09-13T09:08:39.213336+0000","last_fullsized":"2024-09-13T09:08:39.213336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":214,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-13T09:08:39.229216+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:08:39.229216+0000","last_peered":"2024-09-13T09:08:39.229216+0000","last_clean":"2024-09-13T09:08:39.229216+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:08:39.229216+0000","last_undegraded":"2024-09-13T09:08:39.229216+0000","last_fullsized":"2024-09-13T09:08:39.229216+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"89'302","reported_seq":3957,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-09-13T09:08:44.136739+0000","last_change":"2024-09-13T09:07:16.108021+0000","last_active":"2024-09-13T09:08:44.136739+0000","last_peered":"2024-09-13T09:08:44.136739+0000","last_clean":"2024-09-13T09:08:44.136739+0000","last_became_active":"2024-09-13T09:07:13.864063+0000","last_became_peered":"2024-09-13T09:07:13.864063+0000","last_unstale":"2024-09-13T09:08:44.136739+0000","last_undegraded":"2024-09-13T09:08:44.136739+0000","last_fullsized":"2024-09-13T09:08:44.136739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:35.319944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":53315150,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":3968,"num_read_kb":19499,"num_write":447,"num_write_kb":9040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"89'335","reported_seq":2264,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.274734+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:08:40.274734+0000","last_peered":"2024-09-13T09:08:40.274734+0000","last_clean":"2024-09-13T09:08:40.274734+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:08:40.274734+0000","last_undegraded":"2024-09-13T09:08:40.274734+0000","last_fullsized":"2024-09-13T09:08:40.274734+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":335,"log_dups_size":0,"ondisk_log_size":335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":40781830,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":3016,"num_read_kb":10719,"num_write":411,"num_write_kb":5370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"93'398","reported_seq":1935,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T09:08:45.213660+0000","last_change":"2024-09-13T09:08:21.089403+0000","last_active":"2024-09-13T09:08:45.213660+0000","last_peered":"2024-09-13T09:08:45.213660+0000","last_clean":"2024-09-13T09:08:45.213660+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:08:45.213660+0000","last_undegraded":"2024-09-13T09:08:45.213660+0000","last_fullsized":"2024-09-13T09:08:45.213660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":78887234,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":2876,"num_read_kb":20255,"num_write":393,"num_write_kb":16670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'195","reported_seq":1711,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.394424+0000","last_change":"2024-09-13T09:07:16.095554+0000","last_active":"2024-09-13T09:08:40.394424+0000","last_peered":"2024-09-13T09:08:40.394424+0000","last_clean":"2024-09-13T09:08:40.394424+0000","last_became_active":"2024-09-13T09:07:13.866366+0000","last_became_peered":"2024-09-13T09:07:13.866366+0000","last_unstale":"2024-09-13T09:08:40.394424+0000","last_undegraded":"2024-09-13T09:08:40.394424+0000","last_fullsized":"2024-09-13T09:08:40.394424+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:14.596037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":17506740,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":5,"num_read":1508,"num_read_kb":20142,"num_write":224,"num_write_kb":13857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"89'191","reported_seq":16218,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.348655+0000","last_change":"2024-09-13T09:08:38.210244+0000","last_active":"2024-09-13T09:08:40.348655+0000","last_peered":"2024-09-13T09:08:40.348655+0000","last_clean":"2024-09-13T09:08:40.348655+0000","last_became_active":"2024-09-13T09:08:38.208770+0000","last_became_peered":"2024-09-13T09:08:38.208770+0000","last_unstale":"2024-09-13T09:08:40.348655+0000","last_undegraded":"2024-09-13T09:08:40.348655+0000","last_fullsized":"2024-09-13T09:08:40.348655+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:51:52.796334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":48394300,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":10,"num_read":16181,"num_read_kb":25682,"num_write":232,"num_write_kb":13773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":13088571,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'225","reported_seq":2340,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.337061+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:08:40.337061+0000","last_peered":"2024-09-13T09:08:40.337061+0000","last_clean":"2024-09-13T09:08:40.337061+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:08:40.337061+0000","last_undegraded":"2024-09-13T09:08:40.337061+0000","last_fullsized":"2024-09-13T09:08:40.337061+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":18799428,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":2610,"num_read_kb":7327,"num_write":224,"num_write_kb":3960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"89'257","reported_seq":4350,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.264839+0000","last_change":"2024-09-13T09:08:38.205741+0000","last_active":"2024-09-13T09:08:40.264839+0000","last_peered":"2024-09-13T09:08:40.264839+0000","last_clean":"2024-09-13T09:08:40.264839+0000","last_became_active":"2024-09-13T09:08:38.205194+0000","last_became_peered":"2024-09-13T09:08:38.205194+0000","last_unstale":"2024-09-13T09:08:40.264839+0000","last_undegraded":"2024-09-13T09:08:40.264839+0000","last_fullsized":"2024-09-13T09:08:40.264839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:14.407799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":30925341,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":4487,"num_read_kb":10396,"num_write":288,"num_write_kb":2397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"89'202","reported_seq":3013,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.312204+0000","last_change":"2024-09-13T09:08:38.210227+0000","last_active":"2024-09-13T09:08:40.312204+0000","last_peered":"2024-09-13T09:08:40.312204+0000","last_clean":"2024-09-13T09:08:40.312204+0000","last_became_active":"2024-09-13T09:08:38.209645+0000","last_became_peered":"2024-09-13T09:08:38.209645+0000","last_unstale":"2024-09-13T09:08:40.312204+0000","last_undegraded":"2024-09-13T09:08:40.312204+0000","last_fullsized":"2024-09-13T09:08:40.312204+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:25:11.265318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":21340971,"num_objects":27,"num_object_clones":4,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":2980,"num_read_kb":8404,"num_write":201,"num_write_kb":3227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":91,"num_bytes_recovered":2517416,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"89'291","reported_seq":2004,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.464027+0000","last_change":"2024-09-13T09:08:09.863895+0000","last_active":"2024-09-13T09:08:40.464027+0000","last_peered":"2024-09-13T09:08:40.464027+0000","last_clean":"2024-09-13T09:08:40.464027+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:08:40.464027+0000","last_undegraded":"2024-09-13T09:08:40.464027+0000","last_fullsized":"2024-09-13T09:08:40.464027+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":55331104,"num_objects":37,"num_object_clones":3,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":7,"num_read":1969,"num_read_kb":14485,"num_write":301,"num_write_kb":8672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"89'671","reported_seq":2954,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.262969+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:08:40.262969+0000","last_peered":"2024-09-13T09:08:40.262969+0000","last_clean":"2024-09-13T09:08:40.262969+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:08:40.262969+0000","last_undegraded":"2024-09-13T09:08:40.262969+0000","last_fullsized":"2024-09-13T09:08:40.262969+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":22836943,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":5206,"num_read_kb":9462,"num_write":1019,"num_write_kb":4424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"94'293","reported_seq":2657,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-13T09:08:45.205111+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:08:45.205111+0000","last_peered":"2024-09-13T09:08:45.205111+0000","last_clean":"2024-09-13T09:08:45.205111+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:08:45.205111+0000","last_undegraded":"2024-09-13T09:08:45.205111+0000","last_fullsized":"2024-09-13T09:08:45.205111+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":36945354,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":3383,"num_read_kb":8535,"num_write":387,"num_write_kb":7155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"89'345","reported_seq":1498,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-13T09:08:40.264468+0000","last_change":"2024-09-13T09:07:15.880613+0000","last_active":"2024-09-13T09:08:40.264468+0000","last_peered":"2024-09-13T09:08:40.264468+0000","last_clean":"2024-09-13T09:08:40.264468+0000","last_became_active":"2024-09-13T09:07:13.858603+0000","last_became_peered":"2024-09-13T09:07:13.858603+0000","last_unstale":"2024-09-13T09:08:40.264468+0000","last_undegraded":"2024-09-13T09:08:40.264468+0000","last_fullsized":"2024-09-13T09:08:40.264468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:59.025626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":15681485,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":2061,"num_read_kb":7432,"num_write":418,"num_write_kb":6114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1214936474,"num_objects":945,"num_object_clones":206,"num_object_copies":1890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":945,"num_whiteouts":255,"num_read":147923,"num_read_kb":487560,"num_write":14762,"num_write_kb":264042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":901,"num_bytes_recovered":120233803,"num_keys_recovered":268,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":305483776,"data_stored":325602538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11644,"ondisk_log_size":11644,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70364,"kb_used_data":31056,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94301476,"statfs":{"total":96636764160,"available":96564711424,"internally_reserved":0,"allocated":31801344,"data_stored":33219122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.423,"5min":2.423,"15min":2.423},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.91700000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.84499999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.88},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.72699999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118032,"kb_used_data":68872,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94253808,"statfs":{"total":96636764160,"available":96515899392,"internally_reserved":0,"allocated":70524928,"data_stored":76859868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.39300000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.51400000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.502}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.2150000000000001}]}]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79940,"kb_used_data":36540,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94291900,"statfs":{"total":96636764160,"available":96554905600,"internally_reserved":0,"allocated":37416960,"data_stored":37606933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.379}]},{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.254}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.625}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.39300000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.66700000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.52700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79288,"kb_used_data":39980,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94292552,"statfs":{"total":96636764160,"available":96555573248,"internally_reserved":0,"allocated":40939520,"data_stored":43850039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.39600000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.37},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.432}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":1.121},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.92100000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.498},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.2290000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":2.3929999999999998,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.69399999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43624,"kb_used_data":4308,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94328216,"statfs":{"total":96636764160,"available":96592093184,"internally_reserved":0,"allocated":4411392,"data_stored":4135752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.255}]},{"osd":2,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":3.855,"5min":3.855,"15min":3.855},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.6509999999999998,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":2.4860000000000002,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.008}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.875},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.85199999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73948,"kb_used_data":30544,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94297892,"statfs":{"total":96636764160,"available":96561041408,"internally_reserved":0,"allocated":31277056,"data_stored":34987098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44431815},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.871}]},{"osd":1,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":1.1639999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.748},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.91900000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.052},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.1160000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.252},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.0900000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.2130000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66732,"kb_used_data":39708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94305108,"statfs":{"total":96636764160,"available":96568430592,"internally_reserved":0,"allocated":40660992,"data_stored":42654234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":1.121}]},{"osd":1,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":1.0449999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":1.1579999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":1.2709999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.212}]},{"osd":6,"last update":"Fri Sep 13 09:07:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.347},{"interface":"front","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.80500000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.3009999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79192,"kb_used_data":52172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94292648,"statfs":{"total":96636764160,"available":96555671552,"internally_reserved":0,"allocated":53424128,"data_stored":52666449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:07:59 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.998},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":7.9740000000000002,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.79100000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":8.1400000000000006,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":7.9050000000000002,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":1.177},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.91900000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":8.0850000000000009,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.246}]},{"osd":6,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.149}]},{"osd":7,"last update":"Fri Sep 13 09:07:57 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":8.1890000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.2130000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3366912,"data_stored":3656794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":39301120,"data_stored":42780713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36372480,"data_stored":37127975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69476352,"data_stored":76380891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":39600128,"data_stored":42182134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":51761152,"data_stored":51584964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":34308096,"data_stored":38616852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":31297536,"data_stored":33272215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:08:47.089 INFO:tasks.thrashosds.thrasher:Setting 3.1e to [0, 5, 3, 6] 2024-09-13T09:08:47.089 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1e', '0', '5', '3', '6'] 2024-09-13T09:08:47.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1e 0 5 3 6 2024-09-13T09:08:47.127 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:47.127+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:47.130 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:47.130 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:47.130 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:47.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:47.239 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:47.239+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:47.297 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:47.295+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:47.320 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:47.320 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:47.320 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:47.320 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:47.320 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:47.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:47.398 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:47.393+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375683991999999, 2024-09-13T09:08:47.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:47.496 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375663224999997, 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:08:47.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375645372999998, 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:08:47.505 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375628806999998, 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.506 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:47.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.510 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375603222000002, 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:08:47.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375584054999997, 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:08:47.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:08:47.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375566771999999, 2024-09-13T09:08:47.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:47.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:08:47.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375037902000003, 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:08:47.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:08:47.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.375005997000002, 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:08:47.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:08:47.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.374981048999999, 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:08:47.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:47.539 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.374676483999998, 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:47.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:08:47.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.372914061000003, 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:08:47.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.372893210999997, 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:08:47.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:08:47.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.372870886999998, 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:08:47.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.372852197999997, 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:08:47.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.372835146, 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:08:47.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.372818121999998, 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:08:47.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:08:47.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:47.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.362805135000002, 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:47.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:08:47.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:08:47.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:47.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9826506120000005, 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:08:47.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.582 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9824701899999999, 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:47.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:47.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:08:47.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:47.588 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:47.589 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:47.589 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:47.587+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:47.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:47.690 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:47.685+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:47.762 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:47.762 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:47.762 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:47.762 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:47.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:47.819 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:47.819 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:47.819 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:47.819+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:47.835 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:47.835 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:47.851 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:47.851 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:47.867 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:47.867 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:47.886 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.1e pg_upmap_items mapping to [0->5,3->6] 2024-09-13T09:08:47.891 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:47.891+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:47.893 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:47.894 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:47.899 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:08:47.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:47.993 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:47.989+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:47.998 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone32 from 0x93fc7f1 (overlap 0xa639e4) to 0x959a80 2024-09-13T09:08:48.094 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:48.089+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:48.094 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:48.095 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:48.107 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:48.107 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:48.114 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:48.115 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:48.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:48.134 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-13T09:08:48.136 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:48.136 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:48.144 INFO:teuthology.orchestra.run.smithi031.stdout:4353 read 0x4096879 thru 0x409b4dd (0x4c65 bytes) 2024-09-13T09:08:48.162 INFO:teuthology.orchestra.run.smithi031.stdout:4354 write 0x97ffb87 thru 0x9802e90 (0x330a bytes) 2024-09-13T09:08:48.165 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:48.175 INFO:teuthology.orchestra.run.smithi031.stdout:4355 trunc from 0x9802e91 to 0xc9d6c62 2024-09-13T09:08:48.186 INFO:teuthology.orchestra.run.smithi031.stdout:4356 write 0x22033cb thru 0x22054e9 (0x211f bytes) 2024-09-13T09:08:48.203 INFO:teuthology.orchestra.run.smithi031.stdout:4357 trunc from 0xc9d6c62 to 0x962b98e 2024-09-13T09:08:48.204 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:48.203+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:48.217 INFO:teuthology.orchestra.run.smithi031.stdout:4361 writesame 0xc7cdab thru 0xc84f13 data_size 0x1b(0x8169 bytes) 2024-09-13T09:08:48.233 INFO:teuthology.orchestra.run.smithi031.stdout:4362 write 0x3f38ebf thru 0x3f44d2a (0xbe6c bytes) 2024-09-13T09:08:48.246 INFO:teuthology.orchestra.run.smithi031.stdout:4364 clone 34 order 22 su 1048576 sc 9 2024-09-13T09:08:48.258 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:48.258 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:48.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:48.315 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:48.315+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:48.396 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:48.393+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:48.434 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:48.434 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:48.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:48.497 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:48.493+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:48.570 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:48.571 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:48.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:48.598 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:48.593+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:48.699 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:48.697+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:48.710 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:48.710 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:48.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:48.800 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:48.799+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:48.851 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:48.851 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:48.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:48.893 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:08:48.917 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:48.915+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:49.002 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:48.999+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:49.012 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:49.012 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:49.027 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:49.035 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone33 from 0x962b98e (overlap 0x93fc7f1) to 0x1d98e63 2024-09-13T09:08:49.103 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:49.097+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:49.205 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:49.197+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:49.226 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:49.226 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:49.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:49.337 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:49.333+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:49.403 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-13T09:08:49.407 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:49.401+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:49.413 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:49.413 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:49.425 INFO:teuthology.orchestra.run.smithi031.stdout:4366 write 0x855b980 thru 0x856425c (0x88dd bytes) 2024-09-13T09:08:49.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:49.446 INFO:teuthology.orchestra.run.smithi031.stdout:4367 punch from 0x3a6ebfe to 0x3a79d6d, (0xb16f bytes) 2024-09-13T09:08:49.446 INFO:teuthology.orchestra.run.smithi031.stdout:4368 read 0x54684a0 thru 0x54692ea (0xe4b bytes) 2024-09-13T09:08:49.465 INFO:teuthology.orchestra.run.smithi031.stdout:4369 writesame 0xd878336 thru 0xd87a48a data_size 0x17(0x2155 bytes) 2024-09-13T09:08:49.465 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:49.465 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:49.473 INFO:teuthology.orchestra.run.smithi031.stdout:4372 trunc from 0xd87a48b to 0xa43452e 2024-09-13T09:08:49.515 INFO:teuthology.orchestra.run.smithi031.stdout:4373 write 0xa952e89 thru 0xa95908e (0x6206 bytes) 2024-09-13T09:08:49.520 INFO:teuthology.orchestra.run.smithi031.stdout:4374 write 0x7956d8e thru 0x795ea47 (0x7cba bytes) 2024-09-13T09:08:49.521 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:49.517+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:49.531 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:49.537 INFO:teuthology.orchestra.run.smithi031.stdout:4376 trunc from 0xa95908f to 0xca29f00 2024-09-13T09:08:49.546 INFO:teuthology.orchestra.run.smithi031.stdout:4377 read 0x2a80d47 thru 0x2a8d75c (0xca16 bytes) 2024-09-13T09:08:49.546 INFO:teuthology.orchestra.run.smithi031.stdout:4378 write 0x3e5e70b thru 0x3e6d2fa (0xebf0 bytes) 2024-09-13T09:08:49.559 INFO:teuthology.orchestra.run.smithi031.stdout:4380 read 0x49da387 thru 0x49e3398 (0x9012 bytes) 2024-09-13T09:08:49.559 INFO:teuthology.orchestra.run.smithi031.stdout:4381 write 0xa7278aa thru 0xa7299aa (0x2101 bytes) 2024-09-13T09:08:49.596 INFO:teuthology.orchestra.run.smithi031.stdout:4382 trunc from 0xca29f00 to 0xd50ef 2024-09-13T09:08:49.600 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:49.600 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:49.608 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:49.607+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:49.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:49.657 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:49.680 INFO:teuthology.orchestra.run.smithi031.stdout:4385 write 0x40b5caf thru 0x40bde39 (0x818b bytes) 2024-09-13T09:08:49.708 INFO:teuthology.orchestra.run.smithi031.stdout:4386 write 0x6fcf164 thru 0x6fddc9c (0xeb39 bytes) 2024-09-13T09:08:49.709 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:49.705+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:49.716 INFO:teuthology.orchestra.run.smithi031.stdout:4387 punch from 0xdb0afe to 0xdb7f22, (0x7424 bytes) 2024-09-13T09:08:49.716 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:49.738 INFO:teuthology.orchestra.run.smithi031.stdout:4389 write 0xa74ceaf thru 0xa75678d (0x98df bytes) 2024-09-13T09:08:49.745 INFO:teuthology.orchestra.run.smithi031.stdout:4390 writesame 0x212fb26 thru 0x2132b2d data_size 0x8(0x3008 bytes) 2024-09-13T09:08:49.766 INFO:teuthology.orchestra.run.smithi031.stdout:4391 writesame 0xab907fc thru 0xaba0293 data_size 0xa2(0xfa98 bytes) 2024-09-13T09:08:49.767 INFO:teuthology.orchestra.run.smithi031.stdout:4392 read 0x7f3a1e3 thru 0x7f3e870 (0x468e bytes) 2024-09-13T09:08:49.790 INFO:teuthology.orchestra.run.smithi031.stdout:4394 trunc from 0xaba0294 to 0x1ddf92b 2024-09-13T09:08:49.792 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:49.793 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:49.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:49.810 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:49.807+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:49.846 INFO:teuthology.orchestra.run.smithi031.stdout:4398 trunc from 0x1ddf92b to 0xd09ae86 2024-09-13T09:08:49.858 INFO:teuthology.orchestra.run.smithi031.stdout:4399 read 0x3889637 thru 0x3899129 (0xfaf3 bytes) 2024-09-13T09:08:49.858 INFO:teuthology.orchestra.run.smithi031.stdout:4400 writesame 0x8bf00d thru 0x8cb92c data_size 0x10(0xc920 bytes) 2024-09-13T09:08:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:4401 read 0x433f00 thru 0x443744 (0xf845 bytes) 2024-09-13T09:08:49.870 INFO:teuthology.orchestra.run.smithi031.stdout:4402 punch from 0x5831d0e to 0x583d67e, (0xb970 bytes) 2024-09-13T09:08:49.870 INFO:teuthology.orchestra.run.smithi031.stdout:4403 read 0x8a076e4 thru 0x8a0df6e (0x688b bytes) 2024-09-13T09:08:49.870 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:49.870 INFO:teuthology.orchestra.run.smithi031.stdout:4406 read 0x568fecb thru 0x569f9de (0xfb14 bytes) 2024-09-13T09:08:49.870 INFO:teuthology.orchestra.run.smithi031.stdout:4408 read 0x7b69588 thru 0x7b74e00 (0xb879 bytes) 2024-09-13T09:08:49.870 INFO:teuthology.orchestra.run.smithi031.stdout:4409 read 0x7f255a9 thru 0x7f2ce4b (0x78a3 bytes) 2024-09-13T09:08:49.871 INFO:teuthology.orchestra.run.smithi031.stdout:4410 punch from 0x39d1945 to 0x39d8f81, (0x763c bytes) 2024-09-13T09:08:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:4411 writesame 0xd8162df thru 0xd82352f data_size 0x3(0xd251 bytes) 2024-09-13T09:08:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:4416 read 0x32af420 thru 0x32bc8dc (0xd4bd bytes) 2024-09-13T09:08:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:4417 read 0x173aa68 thru 0x17475e9 (0xcb82 bytes) 2024-09-13T09:08:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:4418 punch from 0xd283aca to 0xd28cc57, (0x918d bytes) 2024-09-13T09:08:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:4419 punch from 0x620e9da to 0x62146e4, (0x5d0a bytes) 2024-09-13T09:08:49.884 INFO:teuthology.orchestra.run.smithi031.stdout:4421 read 0x3fca449 thru 0x3fcad79 (0x931 bytes) 2024-09-13T09:08:49.911 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:49.911+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:49.919 INFO:teuthology.orchestra.run.smithi031.stdout:4422 trunc from 0xd823530 to 0xb28d39 2024-09-13T09:08:49.933 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:49.933 INFO:teuthology.orchestra.run.smithi031.stdout:4424 read 0x73aea2 thru 0x73f4d7 (0x4636 bytes) 2024-09-13T09:08:49.958 INFO:teuthology.orchestra.run.smithi031.stdout:4425 write 0x516bcaf thru 0x517954b (0xd89d bytes) 2024-09-13T09:08:49.972 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:49.973 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:49.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:50.000 INFO:teuthology.orchestra.run.smithi031.stdout:4426 writesame 0xe6f3f63 thru 0xe6ff2ef data_size 0x1d(0xb38d bytes) 2024-09-13T09:08:50.000 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:50.000 INFO:teuthology.orchestra.run.smithi031.stdout:4428 writesame 0x6798f56 thru 0x6798fee data_size 0x3(0x99 bytes) 2024-09-13T09:08:50.006 INFO:teuthology.orchestra.run.smithi031.stdout:4429 read 0x3d5babe thru 0x3d5f12f (0x3672 bytes) 2024-09-13T09:08:50.006 INFO:teuthology.orchestra.run.smithi031.stdout:4430 write 0xdde18b6 thru 0xdde7098 (0x57e3 bytes) 2024-09-13T09:08:50.012 INFO:teuthology.orchestra.run.smithi031.stdout:4431 write 0xcfb07a3 thru 0xcfb1367 (0xbc5 bytes) 2024-09-13T09:08:50.033 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:50.029+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:50.055 INFO:teuthology.orchestra.run.smithi031.stdout:4432 trunc from 0xe6ff2f0 to 0x6757 2024-09-13T09:08:50.090 INFO:teuthology.orchestra.run.smithi031.stdout:4433 punch from 0x88e to 0x6757, (0x5ec9 bytes) 2024-09-13T09:08:50.090 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:50.090 INFO:teuthology.orchestra.run.smithi031.stdout:4435 read 0x2233 thru 0x35bc (0x138a bytes) 2024-09-13T09:08:50.091 INFO:teuthology.orchestra.run.smithi031.stdout:4436 punch from 0x5e54 to 0x6757, (0x903 bytes) 2024-09-13T09:08:50.091 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:50.091 INFO:teuthology.orchestra.run.smithi031.stdout:4439 punch from 0x24de to 0x4be7, (0x2709 bytes) 2024-09-13T09:08:50.091 INFO:teuthology.orchestra.run.smithi031.stdout:4441 read 0x13d3 thru 0x6756 (0x5384 bytes) 2024-09-13T09:08:50.093 INFO:teuthology.orchestra.run.smithi031.stdout:4444 trunc from 0x6757 to 0x86549f 2024-09-13T09:08:50.105 INFO:teuthology.orchestra.run.smithi031.stdout:4445 punch from 0x3554cb to 0x35b5bc, (0x60f1 bytes) 2024-09-13T09:08:50.113 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:50.109+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:50.113 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:50.113 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:50.128 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:50.129 INFO:teuthology.orchestra.run.smithi031.stdout:4447 write 0x4bf2549 thru 0x4bfe664 (0xc11c bytes) 2024-09-13T09:08:50.134 INFO:teuthology.orchestra.run.smithi031.stdout:4448 punch from 0xd9cb9b to 0xdaae15, (0xe27a bytes) 2024-09-13T09:08:50.135 INFO:teuthology.orchestra.run.smithi031.stdout:4449 writesame 0x3b31885 thru 0x3b38866 data_size 0x1(0x6fe2 bytes) 2024-09-13T09:08:50.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:50.210 INFO:teuthology.orchestra.run.smithi031.stdout:4450 write 0xe93912e thru 0xe942b08 (0x99db bytes) 2024-09-13T09:08:50.213 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:50.211+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:50.217 INFO:teuthology.orchestra.run.smithi031.stdout:4451 writesame 0xe090093 thru 0xe09e75a data_size 0x38(0xe6c8 bytes) 2024-09-13T09:08:50.221 INFO:teuthology.orchestra.run.smithi031.stdout:4453 read 0x5672f75 thru 0x567b37a (0x8406 bytes) 2024-09-13T09:08:50.221 INFO:teuthology.orchestra.run.smithi031.stdout:4454 flatten 2024-09-13T09:08:50.232 INFO:teuthology.orchestra.run.smithi031.stdout:4455 read 0xa93867 thru 0xa9834f (0x4ae9 bytes) 2024-09-13T09:08:50.232 INFO:teuthology.orchestra.run.smithi031.stdout:4456 write 0x636765 thru 0x637d38 (0x15d4 bytes) 2024-09-13T09:08:50.238 INFO:teuthology.orchestra.run.smithi031.stdout:4457 write 0xe181321 thru 0xe18575d (0x443d bytes) 2024-09-13T09:08:50.244 INFO:teuthology.orchestra.run.smithi031.stdout:4458 punch from 0x2131b20 to 0x2139669, (0x7b49 bytes) 2024-09-13T09:08:50.244 INFO:teuthology.orchestra.run.smithi031.stdout:4459 read 0xe05690c thru 0xe05bf38 (0x562d bytes) 2024-09-13T09:08:50.245 INFO:teuthology.orchestra.run.smithi031.stdout:4461 read 0xbf144f4 thru 0xbf1aaf6 (0x6603 bytes) 2024-09-13T09:08:50.245 INFO:teuthology.orchestra.run.smithi031.stdout:4462 punch from 0x7866b64 to 0x786f33a, (0x87d6 bytes) 2024-09-13T09:08:50.245 INFO:teuthology.orchestra.run.smithi031.stdout:4463 read 0x14074bb thru 0x140d879 (0x63bf bytes) 2024-09-13T09:08:50.245 INFO:teuthology.orchestra.run.smithi031.stdout:4464 writesame 0xa36bf99 thru 0xa379c1b data_size 0x1f(0xdc83 bytes) 2024-09-13T09:08:50.253 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:50.254 INFO:teuthology.orchestra.run.smithi031.stdout:4467 punch from 0xb90d96c to 0xb90e68f, (0xd23 bytes) 2024-09-13T09:08:50.254 INFO:teuthology.orchestra.run.smithi031.stdout:4468 punch from 0x9572316 to 0x9582292, (0xff7c bytes) 2024-09-13T09:08:50.254 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:50.254 INFO:teuthology.orchestra.run.smithi031.stdout:4471 read 0x5ac8b8b thru 0x5acc698 (0x3b0e bytes) 2024-09-13T09:08:50.254 INFO:teuthology.orchestra.run.smithi031.stdout:4472 write 0xe8c0df2 thru 0xe8cbc12 (0xae21 bytes) 2024-09-13T09:08:50.260 INFO:teuthology.orchestra.run.smithi031.stdout:4473 read 0x6035bf2 thru 0x6043632 (0xda41 bytes) 2024-09-13T09:08:50.260 INFO:teuthology.orchestra.run.smithi031.stdout:4474 read 0x8aefd42 thru 0x8af6ac8 (0x6d87 bytes) 2024-09-13T09:08:50.260 INFO:teuthology.orchestra.run.smithi031.stdout:4475 writesame 0x521d0c2 thru 0x5229115 data_size 0x21(0xc054 bytes) 2024-09-13T09:08:50.268 INFO:teuthology.orchestra.run.smithi031.stdout:4476 write 0x44a8ce8 thru 0x44b0ee4 (0x81fd bytes) 2024-09-13T09:08:50.271 INFO:teuthology.orchestra.run.smithi031.stdout:4478 write 0x91cfaed thru 0x91d0b34 (0x1048 bytes) 2024-09-13T09:08:50.274 INFO:teuthology.orchestra.run.smithi031.stdout:4480 read 0x2365a0a thru 0x2367139 (0x1730 bytes) 2024-09-13T09:08:50.274 INFO:teuthology.orchestra.run.smithi031.stdout:4482 write 0x5e37ee2 thru 0x5e40fef (0x910e bytes) 2024-09-13T09:08:50.280 INFO:teuthology.orchestra.run.smithi031.stdout:4483 read 0xb4be2ad thru 0xb4c0782 (0x24d6 bytes) 2024-09-13T09:08:50.281 INFO:teuthology.orchestra.run.smithi031.stdout:4484 read 0x3805189 thru 0x380795b (0x27d3 bytes) 2024-09-13T09:08:50.281 INFO:teuthology.orchestra.run.smithi031.stdout:4485 writesame 0x1565cba thru 0x156bcaf data_size 0x2(0x5ff6 bytes) 2024-09-13T09:08:50.301 INFO:teuthology.orchestra.run.smithi031.stdout:4486 write 0x26cc7ab thru 0x26d726a (0xaac0 bytes) 2024-09-13T09:08:50.309 INFO:teuthology.orchestra.run.smithi031.stdout:4487 write 0xb980342 thru 0xb987f83 (0x7c42 bytes) 2024-09-13T09:08:50.310 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:50.310 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:50.314 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:50.311+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:50.316 INFO:teuthology.orchestra.run.smithi031.stdout:4488 punch from 0x99d88ab to 0x99d994e, (0x10a3 bytes) 2024-09-13T09:08:50.316 INFO:teuthology.orchestra.run.smithi031.stdout:4489 read 0x9c3e12f thru 0x9c4c031 (0xdf03 bytes) 2024-09-13T09:08:50.317 INFO:teuthology.orchestra.run.smithi031.stdout:4490 read 0x4d71d1d thru 0x4d749d8 (0x2cbc bytes) 2024-09-13T09:08:50.317 INFO:teuthology.orchestra.run.smithi031.stdout:4491 read 0x48e35db thru 0x48ea2b3 (0x6cd9 bytes) 2024-09-13T09:08:50.318 INFO:teuthology.orchestra.run.smithi031.stdout:4492 trunc from 0xe942b09 to 0xd54b6f9 2024-09-13T09:08:50.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:50.335 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:50.335 INFO:teuthology.orchestra.run.smithi031.stdout:4495 clone 35 order 24 su 2097152 sc 4 2024-09-13T09:08:50.344 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:50.345 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:50.360 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:50.361 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:50.377 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:50.377 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:50.393 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:50.393 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:50.412 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:50.413 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:50.415 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:50.409+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:50.430 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:50.430 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:50.450 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:50.450 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:50.450 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:50.451 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:50.451 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:50.451 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:50.451 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:50.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:50.470 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:50.471 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:50.531 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:50.531+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:50.616 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:50.615+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193270408999993, 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:08:50.635 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:50.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:08:50.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193245825999995, 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.640 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:08:50.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193221821999998, 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.645 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:08:50.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:50.649 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193198339000006, 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:50.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:08:50.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193174932000005, 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.654 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:50.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:08:50.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193151682999996, 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:08:50.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:50.664 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193119577000004, 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193083959999996, 2024-09-13T09:08:50.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:08:50.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.193051980000007, 2024-09-13T09:08:50.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:50.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:08:50.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192982701000005, 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:08:50.679 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:08:50.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:08:50.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192957641000007, 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.684 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:50.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:08:50.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192934210999994, 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.689 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:08:50.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.693 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192910827999995, 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:08:50.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:08:50.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.698 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192887412000005, 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:08:50.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:08:50.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:50.703 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192863489000004, 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:08:50.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192830404999995, 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.708 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192806605000001, 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.713 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:08:50.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:50.717 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192782278999999, 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:08:50.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:50.722 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192749719000005, 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:50.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:08:50.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 75.192709846, 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:50.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:08:50.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:50.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:50.732 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:50.732 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:50.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:50.734 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:50.731+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:50.834 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:50.829+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:50.862 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:50.863 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:50.863 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:50.863 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:50.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:50.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:08:50.963 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:50.963+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:51.036 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:51.029+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:51.049 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:51.049 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:51.050 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:51.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:51.157 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:51.153+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:51.199 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-13T09:08:51.237 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:51.233+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:51.263 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:51.263 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:51.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:51.263 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:51.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.263 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.263 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037468017999998, 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037437167999997, 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:08:51.268 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:08:51.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037411000999995, 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.273 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:08:51.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:08:51.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:08:51.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.278 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037371527000005, 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:51.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:08:51.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037336972999995, 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:08:51.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:08:51.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.289 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037304617000004, 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037272345999995, 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.294 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:08:51.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:08:51.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037238310000006, 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:08:51.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:08:51.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037205045999997, 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:08:51.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.037172806000001, 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:08:51.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:08:51.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:51.313 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.028192720000007, 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:08:51.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:08:51.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.028151840000007, 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:08:51.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.322 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.028111572, 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.028064529000005, 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:51.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:08:51.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.028018966999994, 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.332 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:08:51.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.027928798999994, 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.337 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:08:51.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:08:51.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.027882245000001, 2024-09-13T09:08:51.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:08:51.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.026241321000001, 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.346 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:08:51.350 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.026175515000006, 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.351 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.355 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.343167869999998, 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:51.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:08:51.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:51.362 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:51.362 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:51.363 INFO:teuthology.orchestra.run.smithi031.stdout:4496 trunc from 0xd54b6f9 to 0x8306040 2024-09-13T09:08:51.363 INFO:teuthology.orchestra.run.smithi031.stdout:4498 punch from 0x47404ae to 0x474da68, (0xd5ba bytes) 2024-09-13T09:08:51.364 INFO:teuthology.orchestra.run.smithi031.stdout:4499 read 0x6070fcc thru 0x607e502 (0xd537 bytes) 2024-09-13T09:08:51.364 INFO:teuthology.orchestra.run.smithi031.stdout:4500 writesame 0x7688202 thru 0x768b717 data_size 0x1e(0x3516 bytes) 2024-09-13T09:08:51.364 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:51.364 INFO:teuthology.orchestra.run.smithi031.stdout:4504 write 0xbcdcd44 thru 0xbce5c5c (0x8f19 bytes) 2024-09-13T09:08:51.364 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:51.357+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:51.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:51.367 INFO:teuthology.orchestra.run.smithi031.stdout:4505 write 0xdc81674 thru 0xdc8c098 (0xaa25 bytes) 2024-09-13T09:08:51.370 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:51.370 INFO:teuthology.orchestra.run.smithi031.stdout:4509 read 0x2da4ac6 thru 0x2dadfff (0x953a bytes) 2024-09-13T09:08:51.370 INFO:teuthology.orchestra.run.smithi031.stdout:4510 read 0x9976d57 thru 0x9985cca (0xef74 bytes) 2024-09-13T09:08:51.371 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:51.371 INFO:teuthology.orchestra.run.smithi031.stdout:4512 read 0x12621cf thru 0x126d05a (0xae8c bytes) 2024-09-13T09:08:51.372 INFO:teuthology.orchestra.run.smithi031.stdout:4513 clone 36 order 20 su 8192 sc 8 2024-09-13T09:08:51.464 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:51.463+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:51.490 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:51.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:51.490 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:51.490 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:51.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:51.591 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:51.591+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:51.666 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:51.663+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:51.677 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:51.677 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:51.677 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:51.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:51.779 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:51.779+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:51.864 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:51.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.748673320000002, 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:08:51.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.748652552999999, 2024-09-13T09:08:51.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:08:51.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.748634701, 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:08:51.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.748618135000001, 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.748592549999998, 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:51.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:08:51.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.748573383, 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:08:51.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:08:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:08:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:08:51.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.748556100000002, 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:51.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:08:51.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.748027229999998, 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:51.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.747995324999998, 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:51.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:08:51.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.747970377000001, 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.747665812000001, 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.745903388999999, 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:08:51.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.745882539, 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:51.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.745860215, 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.926 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:08:51.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.745841526, 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.931 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:08:51.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.745824474000003, 2024-09-13T09:08:51.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:08:51.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:08:51.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.745807450000001, 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:08:51.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.735794462999998, 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:08:51.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.35563994, 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:08:51.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:08:51.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:51.955 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.355459518, 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:51.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:08:51.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:51.961 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:51.961 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:51.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:51.963 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:08:51.964 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:51.957+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:51.995 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone35 from 0xdc8c099 (overlap 0x8306040) to 0x5bb7693 2024-09-13T09:08:52.063 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:52.057+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:52.130 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:52.131 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:52.131 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:52.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:52.149 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-13T09:08:52.164 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:52.157+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:52.265 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:52.263+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:52.275 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:52.275 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:52.275 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:52.275 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:52.275 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:52.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:52.383 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:52.383+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.508034535999997, 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:08:52.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:52.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:08:52.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88209 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24748032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:52.469 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225484944000002, 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033308870999999997, 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88209 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235077+0000", 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.22e-06 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235080+0000", 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235083+0000", 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:08:52.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268290+0000", 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033207532999999997 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268342+0000", 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999997e-05 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268385+0000", 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2874999999999999e-05 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88210 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24772608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225476473000001, 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033438816000000003, 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88210 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235084+0000", 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235086+0000", 2024-09-13T09:08:52.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235088+0000", 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235091+0000", 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4839999999999998e-06 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268421+0000", 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033329923999999997 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268479+0000", 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8529999999999997e-05 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268523+0000", 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4048999999999998e-05 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88211 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24797184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225468008, 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033554247000000002, 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88211 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235094+0000", 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2300000000000001e-06 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235097+0000", 2024-09-13T09:08:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.021e-06 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235099+0000", 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268550+0000", 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033450510000000003 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268593+0000", 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3035999999999999e-05 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268647+0000", 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4336999999999998e-05 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88212 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24821760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225459620000002, 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033675415, 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88212 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235102+0000", 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235105+0000", 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235108+0000", 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.367e-06 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268686+0000", 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033577810999999999 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268737+0000", 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1134999999999997e-05 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268777+0000", 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0187e-05 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88213 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24846336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225450416999998, 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033785911000000002, 2024-09-13T09:08:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88213 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235110+0000", 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235112+0000", 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.728e-06 2024-09-13T09:08:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235119+0000", 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235124+0000", 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268806+0000", 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033682479000000001 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268855+0000", 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999998e-05 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268897+0000", 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1915000000000001e-05 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88214 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24870912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225432781999999, 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033897243000000001, 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88214 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235130+0000", 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235139+0000", 2024-09-13T09:08:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235151+0000", 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1856e-05 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268933+0000", 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033782874999999997 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268986+0000", 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2593000000000003e-05 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269026+0000", 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9546999999999999e-05 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88215 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24895488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225407394999998, 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033999110999999999, 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88215 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235153+0000", 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235155+0000", 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8810000000000001e-06 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235159+0000", 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235161+0000", 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:08:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269056+0000", 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033895001000000001 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269109+0000", 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269153+0000", 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88216 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24920064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225397637, 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120667, 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88216 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235173+0000", 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8490000000000001e-06 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235178+0000", 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235180+0000", 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269188+0000", 2024-09-13T09:08:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034007280000000001 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269245+0000", 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6891000000000001e-05 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269284+0000", 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9561999999999998e-05 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88217 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24944640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225378223, 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034230180999999998, 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88217 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235184+0000", 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3120000000000001e-06 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235187+0000", 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235192+0000", 2024-09-13T09:08:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:08:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269313+0000", 2024-09-13T09:08:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120708 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269363+0000", 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9453999999999998e-05 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269413+0000", 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0531000000000001e-05 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88218 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24969216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225366665999999, 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034354619000000003, 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88218 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235195+0000", 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235199+0000", 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:08:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235201+0000", 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269450+0000", 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034248860999999999 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269502+0000", 2024-09-13T09:08:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2519000000000003e-05 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269549+0000", 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6740999999999999e-05 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88219 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24993792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225358125, 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034466610000000002, 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88219 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235204+0000", 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.184e-06 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235207+0000", 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235209+0000", 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269577+0000", 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034368528000000002 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269628+0000", 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0241999999999998e-05 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269670+0000", 2024-09-13T09:08:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1875999999999997e-05 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88220 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25018368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225350274999997, 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034591746999999999, 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88220 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235212+0000", 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0839999999999999e-06 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235215+0000", 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9830000000000001e-06 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235217+0000", 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269707+0000", 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034489989999999998 2024-09-13T09:08:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269762+0000", 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000003e-05 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269803+0000", 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0181e-05 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88221 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25042944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225342191999999, 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034709489000000003, 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88221 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235220+0000", 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0640000000000001e-06 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235223+0000", 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235225+0000", 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269830+0000", 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034604633000000003 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269884+0000", 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067000000000003e-05 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269928+0000", 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4607000000000002e-05 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88222 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25067520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225334046, 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034835487999999998, 2024-09-13T09:08:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88222 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235228+0000", 2024-09-13T09:08:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235231+0000", 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235233+0000", 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269963+0000", 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034730039999999997 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270016+0000", 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2581999999999999e-05 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270063+0000", 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6718000000000002e-05 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88223 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25092096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225325824999999, 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034953250999999998, 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88223 2024-09-13T09:08:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235236+0000", 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.127e-06 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235239+0000", 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0180000000000002e-06 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235241+0000", 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270091+0000", 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034849036 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270137+0000", 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6112999999999997e-05 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270189+0000", 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88224 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25116672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225317625999999, 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035071738999999998, 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88224 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235244+0000", 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235247+0000", 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7750000000000001e-06 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235249+0000", 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270226+0000", 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034977019999999998 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270275+0000", 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8791000000000001e-05 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270315+0000", 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9953999999999997e-05 2024-09-13T09:08:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88225 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25141248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.225309713999998, 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035182109000000003, 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88225 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235252+0000", 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235255+0000", 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235258+0000", 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270337+0000", 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035079359999999997 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270388+0000", 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0986000000000003e-05 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270433+0000", 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5581000000000003e-05 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93981 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [watch reconnect cookie 140193706156944 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:08:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.606462176000001, 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116345170000001, 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93981 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:08:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854096+0000", 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854113+0000", 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6759999999999999e-05 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854146+0000", 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3506000000000001e-05 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854169+0000", 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2331999999999999e-05 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856384+0000", 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022150960000000002 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856397+0000", 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3047999999999999e-05 2024-09-13T09:08:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863781+0000", 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007384276 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863918+0000", 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013628800000000001 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865733+0000", 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018159109999999999 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93982 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.602798243999999, 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0094988519999999, 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93982 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857761+0000", 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857770+0000", 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2109999999999998e-06 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857789+0000", 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9111999999999999e-05 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857800+0000", 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857854+0000", 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4030999999999998e-05 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857866+0000", 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2275e-05 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864650+0000", 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0067835570000001 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864768+0000", 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118019 2024-09-13T09:08:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865240+0000", 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047203799999999998 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.866250+0000", 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001009935 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867165+0000", 2024-09-13T09:08:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091516600000000003 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867219+0000", 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4330999999999999e-05 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867262+0000", 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.265e-05 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.562 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:52.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:52.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:52.564 INFO:teuthology.orchestra.run.smithi031.stdout:4514 writesame 0x3a389f4 thru 0x3a3f21e data_size 0x3(0x682b bytes) 2024-09-13T09:08:52.564 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:52.563+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:52.607 INFO:teuthology.orchestra.run.smithi031.stdout:4515 write 0x81b2b61 thru 0x81b8161 (0x5601 bytes) 2024-09-13T09:08:52.664 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:52.657+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:52.676 INFO:teuthology.orchestra.run.smithi031.stdout:4517 read 0xae7f1c9 thru 0xae88457 (0x928f bytes) 2024-09-13T09:08:52.676 INFO:teuthology.orchestra.run.smithi031.stdout:4518 read 0x64f5d72 thru 0x6505856 (0xfae5 bytes) 2024-09-13T09:08:52.681 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:52.681 INFO:teuthology.orchestra.run.smithi031.stdout:4520 writesame 0x195c8a8 thru 0x196c26a data_size 0x1(0xf9c3 bytes) 2024-09-13T09:08:52.748 INFO:teuthology.orchestra.run.smithi031.stdout:4522 write 0xa6890d4 thru 0xa68f9d9 (0x6906 bytes) 2024-09-13T09:08:52.765 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:52.763+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:52.768 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:52.768 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:52.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.768 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131162 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.768 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754102+0000", 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010293327, 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010317045, 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131162 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754102+0000", 2024-09-13T09:08:52.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754102+0000", 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754101+0000", 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754107+0000", 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4990000000000002e-06 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754113+0000", 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6429999999999998e-06 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754118+0000", 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6929999999999998e-06 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131258 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754753+0000", 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096425439999999994, 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097305020000000002, 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131258 2024-09-13T09:08:52.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754753+0000", 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754753+0000", 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754752+0000", 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754755+0000", 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7669999999999999e-06 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754761+0000", 2024-09-13T09:08:52.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7450000000000001e-06 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754764+0000", 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6850000000000001e-06 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131163 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754123+0000", 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010272549000000001, 2024-09-13T09:08:52.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010419292, 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131163 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754123+0000", 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754123+0000", 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754122+0000", 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754124+0000", 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1399999999999998e-06 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754130+0000", 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1399999999999997e-06 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754136+0000", 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131259 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754766+0000", 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096286410000000003, 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098354619999999997, 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131259 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.779 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754766+0000", 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754766+0000", 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754766+0000", 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754767+0000", 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1310000000000001e-06 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754772+0000", 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.639e-06 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754775+0000", 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5940000000000001e-06 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131164 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754170+0000", 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010224722, 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01048697, 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131164 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754170+0000", 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754170+0000", 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754170+0000", 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754174+0000", 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7869999999999999e-06 2024-09-13T09:08:52.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754182+0000", 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.926e-06 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754186+0000", 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5050000000000001e-06 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131260 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754777+0000", 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096184660000000009, 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0099409049999999999, 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131260 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754777+0000", 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754777+0000", 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754776+0000", 2024-09-13T09:08:52.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754778+0000", 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1379999999999999e-06 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754781+0000", 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0369999999999999e-06 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754783+0000", 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.486e-06 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131165 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754189+0000", 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010205989, 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010585673, 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.789 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131165 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754189+0000", 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754189+0000", 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754189+0000", 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754191+0000", 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.73e-06 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754195+0000", 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1849999999999997e-06 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754198+0000", 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9849999999999998e-06 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131261 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754785+0000", 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096101499999999996, 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010033142, 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131261 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.793 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754785+0000", 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754785+0000", 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754785+0000", 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754788+0000", 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.836e-06 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754791+0000", 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.225e-06 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754793+0000", 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5629999999999999e-06 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131166 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754200+0000", 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010194663, 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010672114, 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131166 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754200+0000", 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754200+0000", 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754200+0000", 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754205+0000", 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0490000000000003e-06 2024-09-13T09:08:52.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754209+0000", 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6059999999999999e-06 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754212+0000", 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131262 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754795+0000", 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095996359999999999, 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010137402, 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131262 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.800 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754795+0000", 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754795+0000", 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754795+0000", 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754797+0000", 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.319e-06 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754802+0000", 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0830000000000003e-06 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754806+0000", 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.33e-06 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131167 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754214+0000", 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010181292, 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010775654000000001, 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131167 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754214+0000", 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754214+0000", 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754214+0000", 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754215+0000", 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3200000000000001e-06 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754218+0000", 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1259999999999998e-06 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754221+0000", 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8569999999999999e-06 2024-09-13T09:08:52.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131263 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754810+0000", 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095851960000000007, 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01023809, 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131263 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754810+0000", 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754810+0000", 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754810+0000", 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754812+0000", 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1780000000000002e-06 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754818+0000", 2024-09-13T09:08:52.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2840000000000003e-06 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754826+0000", 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5050000000000001e-06 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131168 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754224+0000", 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010171483, 2024-09-13T09:08:52.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010884556, 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131168 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754224+0000", 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754224+0000", 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754223+0000", 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754225+0000", 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.872e-06 2024-09-13T09:08:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754231+0000", 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5670000000000002e-06 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754235+0000", 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3100000000000002e-06 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131264 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754831+0000", 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095644270000000003, 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010335357, 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131264 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754831+0000", 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754831+0000", 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754830+0000", 2024-09-13T09:08:52.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754833+0000", 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4770000000000002e-06 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754839+0000", 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0170000000000002e-06 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754842+0000", 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9160000000000001e-06 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131169 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754239+0000", 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010155809999999999, 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010984905999999999, 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.817 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131169 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754239+0000", 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754239+0000", 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754239+0000", 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754241+0000", 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4210000000000002e-06 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754248+0000", 2024-09-13T09:08:52.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3389999999999998e-06 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754252+0000", 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7110000000000003e-06 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131265 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754844+0000", 2024-09-13T09:08:52.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095510760000000004, 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010429974999999999, 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131265 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754844+0000", 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754844+0000", 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754844+0000", 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754845+0000", 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2860000000000001e-06 2024-09-13T09:08:52.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754849+0000", 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7929999999999999e-06 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754852+0000", 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.65e-06 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131170 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754257+0000", 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010138126000000001, 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011046660999999999, 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131170 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754257+0000", 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754257+0000", 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754257+0000", 2024-09-13T09:08:52.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754259+0000", 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6199999999999999e-06 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754266+0000", 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3289999999999999e-06 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754270+0000", 2024-09-13T09:08:52.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.729e-06 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131266 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754854+0000", 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095415510000000005, 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010479819, 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131266 2024-09-13T09:08:52.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754854+0000", 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754854+0000", 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754853+0000", 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754855+0000", 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2470000000000001e-06 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754858+0000", 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-06 2024-09-13T09:08:52.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754864+0000", 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5779999999999998e-06 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131171 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754283+0000", 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010112143000000001, 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011105353, 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131171 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754283+0000", 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754283+0000", 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754282+0000", 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754285+0000", 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754296+0000", 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1195e-05 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754300+0000", 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8160000000000004e-06 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131267 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754889+0000", 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0095060820000000008, 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010558802000000001, 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131267 2024-09-13T09:08:52.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754889+0000", 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754889+0000", 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754888+0000", 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754895+0000", 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4620000000000001e-06 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754902+0000", 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359999999999997e-06 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754909+0000", 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3810000000000004e-06 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131172 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754302+0000", 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010092814, 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011204631, 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131172 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754302+0000", 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754302+0000", 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754302+0000", 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754304+0000", 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.606e-06 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754307+0000", 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3380000000000001e-06 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754311+0000", 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4429999999999996e-06 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131268 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754916+0000", 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094791609999999998, 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010648835000000001, 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131268 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754916+0000", 2024-09-13T09:08:52.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754916+0000", 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754916+0000", 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754918+0000", 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7269999999999999e-06 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754925+0000", 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9120000000000001e-06 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754930+0000", 2024-09-13T09:08:52.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.848e-06 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131173 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754315+0000", 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010080028, 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011307431999999999, 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131173 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754315+0000", 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754315+0000", 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754315+0000", 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754322+0000", 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1119999999999998e-06 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754329+0000", 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8140000000000003e-06 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754336+0000", 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8449999999999997e-06 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131269 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754934+0000", 2024-09-13T09:08:52.847 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094607149999999997, 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01072203, 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131269 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754934+0000", 2024-09-13T09:08:52.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754934+0000", 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754934+0000", 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754936+0000", 2024-09-13T09:08:52.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1679999999999998e-06 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754943+0000", 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3840000000000002e-06 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754947+0000", 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.78e-06 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131174 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754341+0000", 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010054110999999999, 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011343710999999999, 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131174 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754341+0000", 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754341+0000", 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754341+0000", 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754343+0000", 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7180000000000001e-06 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754350+0000", 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4849999999999997e-06 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754354+0000", 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.498e-06 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131270 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.854 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754951+0000", 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094438339999999999, 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010767057999999999, 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131270 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754951+0000", 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754951+0000", 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754951+0000", 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754953+0000", 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2460000000000002e-06 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754964+0000", 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0363999999999999e-05 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754969+0000", 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4160000000000003e-06 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131175 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754357+0000", 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010038282000000001, 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011412031, 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131175 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754357+0000", 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754357+0000", 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754357+0000", 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754358+0000", 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4810000000000001e-06 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754361+0000", 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1379999999999999e-06 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754364+0000", 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2289999999999999e-06 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131271 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754973+0000", 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094218549999999998, 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010849588, 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131271 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754973+0000", 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754973+0000", 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754973+0000", 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754975+0000", 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3350000000000001e-06 2024-09-13T09:08:52.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754981+0000", 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1959999999999996e-06 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754986+0000", 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4360000000000004e-06 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131176 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754383+0000", 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.010011693, 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011480115000000001, 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131176 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754383+0000", 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754383+0000", 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754383+0000", 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754386+0000", 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7819999999999998e-06 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754395+0000", 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9039999999999998e-06 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754408+0000", 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3621999999999999e-05 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131272 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754990+0000", 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0094051199999999995, 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010925219, 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131272 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754990+0000", 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754990+0000", 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754990+0000", 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754992+0000", 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0250000000000001e-06 2024-09-13T09:08:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754998+0000", 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9789999999999998e-06 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755002+0000", 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8169999999999998e-06 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131177 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754415+0000", 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0099802209999999992, 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011552859, 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131177 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754415+0000", 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754415+0000", 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754414+0000", 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754417+0000", 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.683e-06 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754424+0000", 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.776e-06 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754431+0000", 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131273 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755006+0000", 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0093886349999999993, 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010994712, 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131273 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755006+0000", 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755006+0000", 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755006+0000", 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755009+0000", 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0769999999999999e-06 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755017+0000", 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9310000000000008e-06 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755022+0000", 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4510000000000002e-06 2024-09-13T09:08:52.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131178 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754435+0000", 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0099602790000000007, 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011626258, 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131178 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754435+0000", 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754435+0000", 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754435+0000", 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754437+0000", 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1210000000000001e-06 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754450+0000", 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3249e-05 2024-09-13T09:08:52.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754455+0000", 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1850000000000003e-06 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131274 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755026+0000", 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0093694690000000001, 2024-09-13T09:08:52.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011096256000000001, 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131274 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755026+0000", 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755026+0000", 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755025+0000", 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755027+0000", 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1900000000000002e-06 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755042+0000", 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.467e-05 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755052+0000", 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8660000000000001e-06 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131179 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754465+0000", 2024-09-13T09:08:52.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0099300310000000006, 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011718325999999999, 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131179 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754465+0000", 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754465+0000", 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754465+0000", 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754468+0000", 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-06 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754475+0000", 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0609999999999998e-06 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754485+0000", 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0441e-05 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131275 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755057+0000", 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0093382099999999996, 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011200415999999999, 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131275 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755057+0000", 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755057+0000", 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755057+0000", 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755059+0000", 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3369999999999998e-06 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755065+0000", 2024-09-13T09:08:52.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5579999999999997e-06 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755072+0000", 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6619999999999999e-06 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131180 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754490+0000", 2024-09-13T09:08:52.892 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0099053230000000006, 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011797787000000001, 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131180 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754490+0000", 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754490+0000", 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754489+0000", 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754492+0000", 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754497+0000", 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8900000000000004e-06 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754502+0000", 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9289999999999997e-06 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131276 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755078+0000", 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0093175289999999997, 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011238712, 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131276 2024-09-13T09:08:52.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755078+0000", 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755078+0000", 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755077+0000", 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755079+0000", 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2709999999999999e-06 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755086+0000", 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1940000000000003e-06 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755090+0000", 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999999e-06 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131181 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754506+0000", 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0098887909999999992, 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011840745999999999, 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131181 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754506+0000", 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754506+0000", 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754506+0000", 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754508+0000", 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.069e-06 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754514+0000", 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1029999999999998e-06 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754519+0000", 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6500000000000004e-06 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131277 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755143+0000", 2024-09-13T09:08:52.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0092516930000000001, 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011238493, 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131277 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755143+0000", 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755143+0000", 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755143+0000", 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755145+0000", 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8729999999999999e-06 2024-09-13T09:08:52.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755153+0000", 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1990000000000006e-06 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755156+0000", 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1990000000000002e-06 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131182 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754523+0000", 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0098724369999999995, 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011915749999999999, 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131182 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754523+0000", 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754523+0000", 2024-09-13T09:08:52.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754522+0000", 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754524+0000", 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.046e-06 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754530+0000", 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8980000000000001e-06 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754536+0000", 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4419999999999997e-06 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131278 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755159+0000", 2024-09-13T09:08:52.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0092363489999999996, 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01133816, 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131278 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755159+0000", 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755159+0000", 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755159+0000", 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755160+0000", 2024-09-13T09:08:52.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2330000000000001e-06 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755163+0000", 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755166+0000", 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5519999999999999e-06 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131183 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754540+0000", 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0098555569999999992, 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012000798, 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131183 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754540+0000", 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754540+0000", 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754539+0000", 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754541+0000", 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.181e-06 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754549+0000", 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7880000000000007e-06 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754555+0000", 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.066e-06 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131279 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755168+0000", 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0092274320000000007, 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011421501000000001, 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131279 2024-09-13T09:08:52.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755168+0000", 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755168+0000", 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755167+0000", 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755173+0000", 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7690000000000002e-06 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755180+0000", 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5549999999999999e-06 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755184+0000", 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4719999999999997e-06 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131184 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754559+0000", 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0098356840000000008, 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012080963, 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131184 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754559+0000", 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754559+0000", 2024-09-13T09:08:52.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754559+0000", 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754561+0000", 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.12e-06 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754569+0000", 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9559999999999997e-06 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754574+0000", 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1370000000000001e-06 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131280 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755189+0000", 2024-09-13T09:08:52.922 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0092066160000000008, 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011510207, 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131280 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755189+0000", 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755189+0000", 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755188+0000", 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755190+0000", 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1849999999999998e-06 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755196+0000", 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0190000000000003e-06 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755201+0000", 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8160000000000001e-06 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131185 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754579+0000", 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0098162869999999999, 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012178664000000001, 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131185 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754579+0000", 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754579+0000", 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754579+0000", 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754589+0000", 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0753e-05 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754608+0000", 2024-09-13T09:08:52.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9043999999999999e-05 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754615+0000", 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2079999999999997e-06 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131281 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.755205+0000", 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0091899249999999998, 2024-09-13T09:08:52.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011610831, 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131281 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755205+0000", 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755205+0000", 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755205+0000", 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755207+0000", 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1780000000000002e-06 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755212+0000", 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8629999999999999e-06 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.755215+0000", 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9009999999999998e-06 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131250 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754620+0000", 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097748869999999995, 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012254111, 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131250 2024-09-13T09:08:52.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754620+0000", 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754620+0000", 2024-09-13T09:08:52.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754620+0000", 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754623+0000", 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0680000000000001e-06 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754631+0000", 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.439e-06 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754638+0000", 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2709999999999998e-06 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131251 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754644+0000", 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097511669999999998, 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012286162999999999, 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131251 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754644+0000", 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754644+0000", 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754643+0000", 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754646+0000", 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754653+0000", 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8170000000000001e-06 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754658+0000", 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.967e-06 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131252 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754662+0000", 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097329619999999995, 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012324286, 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131252 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754662+0000", 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754662+0000", 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754662+0000", 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754665+0000", 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9349999999999999e-06 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754680+0000", 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5892e-05 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754687+0000", 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2990000000000002e-06 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131253 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754692+0000", 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0097028930000000006, 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012352928000000001, 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131253 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754692+0000", 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754692+0000", 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754692+0000", 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754694+0000", 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0020000000000001e-06 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754703+0000", 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9379999999999998e-06 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754708+0000", 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131254 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754710+0000", 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096853370000000005, 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012391391999999999, 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131254 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754710+0000", 2024-09-13T09:08:52.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754710+0000", 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754710+0000", 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754711+0000", 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.279e-06 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754714+0000", 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0649999999999999e-06 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754716+0000", 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:08:52.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131255 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754719+0000", 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.009676568, 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012439801, 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131255 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754719+0000", 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754719+0000", 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754718+0000", 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754722+0000", 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0779999999999997e-06 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754728+0000", 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2530000000000004e-06 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754730+0000", 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7250000000000002e-06 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131256 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754733+0000", 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096623330000000004, 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012479954999999999, 2024-09-13T09:08:52.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131256 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754733+0000", 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754733+0000", 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754733+0000", 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754734+0000", 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2249999999999999e-06 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754737+0000", 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754740+0000", 2024-09-13T09:08:52.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9919999999999999e-06 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:131257 3.3 3.dad42e03 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e99)", 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:52.754742+0000", 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0096532719999999992, 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.012526692000000001, 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "queued for pg", 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:52.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 131257 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754742+0000", 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754742+0000", 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754742+0000", 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754743+0000", 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.153e-06 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754748+0000", 2024-09-13T09:08:52.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.028e-06 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:52.754751+0000", 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7460000000000001e-06 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 56 2024-09-13T09:08:52.960 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:52.961 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:08:52.961 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:08:52.961 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-13T09:08:52.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-13T09:08:52.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:52.963 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:52.963 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:52.964 INFO:teuthology.orchestra.run.smithi031.stdout:4523 trunc from 0xdc8c099 to 0xcc562db 2024-09-13T09:08:52.964 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:52.964 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:52.964 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:52.965 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:52.965 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:52.965 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:52.965 INFO:teuthology.orchestra.run.smithi031.stdout:4524 write 0x6f4ce88 thru 0x6f519c7 (0x4b40 bytes) 2024-09-13T09:08:52.966 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:52.966 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:52.966 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:52.966 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:52.967 INFO:teuthology.orchestra.run.smithi031.stdout:4525 write 0xb0ea2b0 thru 0xb0ee5c2 (0x4313 bytes) 2024-09-13T09:08:52.967 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:52.967 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:52.968 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:52.968 INFO:teuthology.orchestra.run.smithi031.stdout:4529 read 0x19c4153 thru 0x19d13a6 (0xd254 bytes) 2024-09-13T09:08:52.968 INFO:teuthology.orchestra.run.smithi031.stdout:4530 read 0x6f84570 thru 0x6f92ae6 (0xe577 bytes) 2024-09-13T09:08:52.968 INFO:teuthology.orchestra.run.smithi031.stdout:4531 trunc from 0xcc562db to 0x9920c78 2024-09-13T09:08:52.968 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:52.968 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:52.969 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:52.959+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:53.052 INFO:teuthology.orchestra.run.smithi031.stdout:4532 writesame 0x6bb8772 thru 0x6bc0040 data_size 0x3d(0x78cf bytes) 2024-09-13T09:08:53.062 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:53.059+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:53.081 INFO:teuthology.orchestra.run.smithi031.stdout:4533 write 0x34a9b25 thru 0x34b852b (0xea07 bytes) 2024-09-13T09:08:53.124 INFO:teuthology.orchestra.run.smithi031.stdout:4534 read 0x6b25887 thru 0x6b33b72 (0xe2ec bytes) 2024-09-13T09:08:53.126 INFO:teuthology.orchestra.run.smithi031.stdout:4535 punch from 0x46f13f to 0x47e990, (0xf851 bytes) 2024-09-13T09:08:53.126 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.126 INFO:teuthology.orchestra.run.smithi031.stdout:4538 punch from 0x2915500 to 0x29157d8, (0x2d8 bytes) 2024-09-13T09:08:53.126 INFO:teuthology.orchestra.run.smithi031.stdout:4539 write 0x372bda7 thru 0x373b3f1 (0xf64b bytes) 2024-09-13T09:08:53.130 INFO:teuthology.orchestra.run.smithi031.stdout:4540 writesame 0x5af96b thru 0x5bda0f data_size 0x1(0xe0a5 bytes) 2024-09-13T09:08:53.150 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:53.150 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:53.163 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:53.163+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:53.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:53.182 INFO:teuthology.orchestra.run.smithi031.stdout:4541 punch from 0x2a25394 to 0x2a2cb54, (0x77c0 bytes) 2024-09-13T09:08:53.182 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.182 INFO:teuthology.orchestra.run.smithi031.stdout:4543 punch from 0x278a030 to 0x2798dea, (0xedba bytes) 2024-09-13T09:08:53.182 INFO:teuthology.orchestra.run.smithi031.stdout:4544 punch from 0x2c1713c to 0x2c186f7, (0x15bb bytes) 2024-09-13T09:08:53.182 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.192 INFO:teuthology.orchestra.run.smithi031.stdout:4548 writesame 0xd45ba1b thru 0xd4624c1 data_size 0x13(0x6aa7 bytes) 2024-09-13T09:08:53.193 INFO:teuthology.orchestra.run.smithi031.stdout:4549 read 0x20550a5 thru 0x206400a (0xef66 bytes) 2024-09-13T09:08:53.193 INFO:teuthology.orchestra.run.smithi031.stdout:4550 punch from 0xa011adf to 0xa01a3e6, (0x8907 bytes) 2024-09-13T09:08:53.200 INFO:teuthology.orchestra.run.smithi031.stdout:4551 write 0xe63a5bf thru 0xe644dcf (0xa811 bytes) 2024-09-13T09:08:53.211 INFO:teuthology.orchestra.run.smithi031.stdout:4553 write 0xe1e0314 thru 0xe1e9553 (0x9240 bytes) 2024-09-13T09:08:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:4555 punch from 0x98a8cb2 to 0x98aefa8, (0x62f6 bytes) 2024-09-13T09:08:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:4558 punch from 0xc92984d to 0xc92f5e5, (0x5d98 bytes) 2024-09-13T09:08:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.217 INFO:teuthology.orchestra.run.smithi031.stdout:4560 read 0x636c4fe thru 0x6370c6f (0x4772 bytes) 2024-09-13T09:08:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:4561 read 0xb4826f2 thru 0xb490a07 (0xe316 bytes) 2024-09-13T09:08:53.218 INFO:teuthology.orchestra.run.smithi031.stdout:4562 write 0xcc86421 thru 0xcc94063 (0xdc43 bytes) 2024-09-13T09:08:53.228 INFO:teuthology.orchestra.run.smithi031.stdout:4563 write 0x4b2eb6a thru 0x4b3e04d (0xf4e4 bytes) 2024-09-13T09:08:53.264 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:53.257+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:53.304 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.304 INFO:teuthology.orchestra.run.smithi031.stdout:4565 writesame 0x6653a6a thru 0x665943d data_size 0x2(0x59d4 bytes) 2024-09-13T09:08:53.325 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:53.325 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:53.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:53.344 INFO:teuthology.orchestra.run.smithi031.stdout:4566 read 0x61fa435 thru 0x620421b (0x9de7 bytes) 2024-09-13T09:08:53.345 INFO:teuthology.orchestra.run.smithi031.stdout:4567 punch from 0xc9c0ed4 to 0xc9ca6e0, (0x980c bytes) 2024-09-13T09:08:53.345 INFO:teuthology.orchestra.run.smithi031.stdout:4568 read 0x2cacf83 thru 0x2cbbe87 (0xef05 bytes) 2024-09-13T09:08:53.346 INFO:teuthology.orchestra.run.smithi031.stdout:4569 trunc from 0xe644dd0 to 0xedc6793 2024-09-13T09:08:53.357 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-13T09:08:53.355+0000 7f426cbbd640 -1 mon.a@0(leader).osd e99 definitely_dead 0 2024-09-13T09:08:53.363 INFO:teuthology.orchestra.run.smithi031.stdout:4570 trunc from 0xedc6793 to 0x438c500 2024-09-13T09:08:53.365 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:53.361+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:53.463 INFO:teuthology.orchestra.run.smithi031.stdout:4571 write 0xaf58b80 thru 0xaf66e25 (0xe2a6 bytes) 2024-09-13T09:08:53.465 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:53.463+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:53.475 INFO:teuthology.orchestra.run.smithi031.stdout:4572 writesame 0x49b7c9c thru 0x49b97f5 data_size 0x9(0x1b5a bytes) 2024-09-13T09:08:53.486 INFO:teuthology.orchestra.run.smithi031.stdout:4574 write 0xd70c867 thru 0xd717304 (0xaa9e bytes) 2024-09-13T09:08:53.490 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:53.490 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:53.494 INFO:teuthology.orchestra.run.smithi031.stdout:4575 read 0x1d4a3b7 thru 0x1d4be2c (0x1a76 bytes) 2024-09-13T09:08:53.500 INFO:teuthology.orchestra.run.smithi031.stdout:4577 write 0xd9a6610 thru 0xd9aea66 (0x8457 bytes) 2024-09-13T09:08:53.507 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:53.509 INFO:teuthology.orchestra.run.smithi031.stdout:4578 writesame 0x1dc9d thru 0x1fd82 data_size 0x1(0x20e6 bytes) 2024-09-13T09:08:53.515 INFO:teuthology.orchestra.run.smithi031.stdout:4579 punch from 0xac7eeb6 to 0xac8b3db, (0xc525 bytes) 2024-09-13T09:08:53.515 INFO:teuthology.orchestra.run.smithi031.stdout:4581 read 0x42426c8 thru 0x424e362 (0xbc9b bytes) 2024-09-13T09:08:53.517 INFO:teuthology.orchestra.run.smithi031.stdout:4582 trunc from 0xd9aea67 to 0xed308d6 2024-09-13T09:08:53.523 INFO:teuthology.orchestra.run.smithi031.stdout:4583 read 0xc94d2ab thru 0xc95b291 (0xdfe7 bytes) 2024-09-13T09:08:53.523 INFO:teuthology.orchestra.run.smithi031.stdout:4584 read 0x5f5f537 thru 0x5f61a52 (0x251c bytes) 2024-09-13T09:08:53.524 INFO:teuthology.orchestra.run.smithi031.stdout:4585 write 0x167c8ce thru 0x168c3ec (0xfb1f bytes) 2024-09-13T09:08:53.559 INFO:teuthology.orchestra.run.smithi031.stdout:4586 punch from 0x5556bab to 0x555987a, (0x2ccf bytes) 2024-09-13T09:08:53.559 INFO:teuthology.orchestra.run.smithi031.stdout:4589 read 0xeabd231 thru 0xeac91a9 (0xbf79 bytes) 2024-09-13T09:08:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:4593 read 0x5ba8da7 thru 0x5bb89f9 (0xfc53 bytes) 2024-09-13T09:08:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:4596 write 0x5a57bd thru 0x5b1978 (0xc1bc bytes) 2024-09-13T09:08:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:4600 writesame 0xb465a0 thru 0xb562c1 data_size 0x1(0xfd22 bytes) 2024-09-13T09:08:53.566 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:53.563+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:53.609 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.610 INFO:teuthology.orchestra.run.smithi031.stdout:4602 write 0x8fb3557 thru 0x8fbbeac (0x8956 bytes) 2024-09-13T09:08:53.619 INFO:teuthology.orchestra.run.smithi031.stdout:4604 write 0x35b2758 thru 0x35b56e3 (0x2f8c bytes) 2024-09-13T09:08:53.622 INFO:teuthology.orchestra.run.smithi031.stdout:4605 read 0xb3cc12f thru 0xb3dc037 (0xff09 bytes) 2024-09-13T09:08:53.622 INFO:teuthology.orchestra.run.smithi031.stdout:4606 write 0x5187703 thru 0x518b3e9 (0x3ce7 bytes) 2024-09-13T09:08:53.629 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.629 INFO:teuthology.orchestra.run.smithi031.stdout:4610 read 0x9556aff thru 0x9562474 (0xb976 bytes) 2024-09-13T09:08:53.629 INFO:teuthology.orchestra.run.smithi031.stdout:4611 punch from 0x874c076 to 0x875216b, (0x60f5 bytes) 2024-09-13T09:08:53.629 INFO:teuthology.orchestra.run.smithi031.stdout:4615 punch from 0x44eb8b7 to 0x44ef0d1, (0x381a bytes) 2024-09-13T09:08:53.629 INFO:teuthology.orchestra.run.smithi031.stdout:4616 writesame 0x6172662 thru 0x61796b8 data_size 0x1(0x7057 bytes) 2024-09-13T09:08:53.646 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:53.646 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:53.649 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.649 INFO:teuthology.orchestra.run.smithi031.stdout:4619 writesame 0xd1080c9 thru 0xd116ece data_size 0x1(0xee06 bytes) 2024-09-13T09:08:53.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:53.667 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:53.661+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:53.694 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.694 INFO:teuthology.orchestra.run.smithi031.stdout:4621 writesame 0x4c698d1 thru 0x4c74ac5 data_size 0x1(0xb1f5 bytes) 2024-09-13T09:08:53.721 INFO:teuthology.orchestra.run.smithi031.stdout:4623 write 0x78bb594 thru 0x78c24ce (0x6f3b bytes) 2024-09-13T09:08:53.759 INFO:teuthology.orchestra.run.smithi031.stdout:4624 trunc from 0xed308d6 to 0x13e2094 2024-09-13T09:08:53.770 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:53.765+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:53.818 INFO:teuthology.orchestra.run.smithi031.stdout:4626 write 0xb070918 thru 0xb074a57 (0x4140 bytes) 2024-09-13T09:08:53.826 INFO:teuthology.orchestra.run.smithi031.stdout:4627 punch from 0x26f5c90 to 0x2701686, (0xb9f6 bytes) 2024-09-13T09:08:53.826 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.826 INFO:teuthology.orchestra.run.smithi031.stdout:4629 punch from 0x2ba0d21 to 0x2baa8cf, (0x9bae bytes) 2024-09-13T09:08:53.826 INFO:teuthology.orchestra.run.smithi031.stdout:4630 write 0x55cca79 thru 0x55dbd63 (0xf2eb bytes) 2024-09-13T09:08:53.834 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:53.835 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:53.840 INFO:teuthology.orchestra.run.smithi031.stdout:4631 read 0x6dd10af thru 0x6dd1613 (0x565 bytes) 2024-09-13T09:08:53.840 INFO:teuthology.orchestra.run.smithi031.stdout:4632 write 0x51b2a84 thru 0x51c1108 (0xe685 bytes) 2024-09-13T09:08:53.844 INFO:teuthology.orchestra.run.smithi031.stdout:4633 write 0x3c60959 thru 0x3c674b8 (0x6b60 bytes) 2024-09-13T09:08:53.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:53.854 INFO:teuthology.orchestra.run.smithi031.stdout:4635 write 0x3379aa8 thru 0x337ef54 (0x54ad bytes) 2024-09-13T09:08:53.861 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:53.861 INFO:teuthology.orchestra.run.smithi031.stdout:4638 read 0x519254e thru 0x519f3c1 (0xce74 bytes) 2024-09-13T09:08:53.862 INFO:teuthology.orchestra.run.smithi031.stdout:4639 read 0xa49ee51 thru 0xa49f152 (0x302 bytes) 2024-09-13T09:08:53.862 INFO:teuthology.orchestra.run.smithi031.stdout:4641 punch from 0x16aa434 to 0x16b77b9, (0xd385 bytes) 2024-09-13T09:08:53.862 INFO:teuthology.orchestra.run.smithi031.stdout:4642 writesame 0x741b960 thru 0x741f3e3 data_size 0x1c(0x3a84 bytes) 2024-09-13T09:08:53.867 INFO:teuthology.orchestra.run.smithi031.stdout:4643 read 0x6d9d358 thru 0x6da0f53 (0x3bfc bytes) 2024-09-13T09:08:53.880 INFO:teuthology.orchestra.run.smithi031.stdout:4645 trunc from 0xb074a58 to 0x12411cb 2024-09-13T09:08:53.896 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:53.893+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:53.900 INFO:teuthology.orchestra.run.smithi031.stdout:4646 read 0xb76f91 thru 0xb7a5c1 (0x3631 bytes) 2024-09-13T09:08:53.902 INFO:teuthology.orchestra.run.smithi031.stdout:4647 trunc from 0x12411cb to 0x3af40e 2024-09-13T09:08:53.920 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.7. 2024-09-13T09:08:53.927 INFO:teuthology.orchestra.run.smithi031.stdout:4648 read 0x206d53 thru 0x211779 (0xaa27 bytes) 2024-09-13T09:08:53.947 INFO:teuthology.orchestra.run.smithi031.stdout:4650 trunc from 0x3af40e to 0xe8806a1 2024-09-13T09:08:53.953 INFO:teuthology.orchestra.run.smithi031.stdout:4651 read 0x5beb0f7 thru 0x5bf0b6b (0x5a75 bytes) 2024-09-13T09:08:53.953 INFO:teuthology.orchestra.run.smithi031.stdout:4652 writesame 0x47eaf50 thru 0x47f13ea data_size 0x11(0x649b bytes) 2024-09-13T09:08:53.961 INFO:teuthology.orchestra.run.smithi031.stdout:4654 write 0x34feb82 thru 0x350255d (0x39dc bytes) 2024-09-13T09:08:53.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:08:53.994 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:53.994 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:54.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:54.011 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:54.011+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:54.070 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:54.067+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:54.168 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:54.168 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:54.171 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:54.167+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:54.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:54.272 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:54.265+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:54.310 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:54.310 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:54.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:54.373 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:54.371+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:54.474 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:54.469+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:54.496 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:54.496 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:54.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:54.597 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:54.593+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:54.671 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:54.671 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:54.676 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:54.668+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:54.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:54.777 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:54.772+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:54.857 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:54.857 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:54.872 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:54.878 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:54.875+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:54.924 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:08:54.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:54.978 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:54.972+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:55.037 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:55.037 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:55.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:55.097 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:55.092+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:55.142 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:55.157 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:55.158 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:55.173 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:55.174 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:55.179 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:55.179+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:55.191 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:55.192 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:55.211 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:55.212 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:55.225 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:55.225 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:55.231 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:55.231 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:55.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:55.250 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:55.251 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:55.270 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:55.270 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:55.284 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:55.280+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:55.368 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:55.369 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:55.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:55.369 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:55.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.369 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.369 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925472642000003, 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:08:55.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925448059000004, 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.374 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:08:55.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925424054999993, 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.379 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:08:55.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925400572000001, 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:08:55.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:08:55.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.388 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925377165, 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925353916000006, 2024-09-13T09:08:55.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:55.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:08:55.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.92532181, 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.398 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:08:55.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925286193000005, 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.403 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:08:55.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.407 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925254213000002, 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.408 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:08:55.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925184934000001, 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.413 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.417 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925159874000002, 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.418 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:08:55.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:08:55.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925136444000003, 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.423 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:08:55.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:08:55.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:55.427 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925113061000005, 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:55.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:08:55.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:08:55.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925089645, 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:08:55.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:08:55.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.436 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925065721999999, 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:08:55.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:08:55.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.441 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925032638000005, 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:08:55.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.925008837999997, 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.446 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.450 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.924984511999995, 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.451 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:08:55.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:08:55.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:08:55.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:08:55.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:08:55.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.456 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.457 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:55.457 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.924951952000001, 2024-09-13T09:08:55.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:08:55.457 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.457 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.457 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.457 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.457 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:08:55.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:08:55.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:08:55.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.462 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.462 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:08:55.462 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:55.463 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 79.924912078999995, 2024-09-13T09:08:55.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:08:55.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:55.463 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:55.463 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:55.463 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:55.463 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:55.463 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:08:55.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.464 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:55.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:55.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:55.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:55.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:55.468 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:55.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:08:55.471 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:55.467+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:55.572 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:55.564+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:55.644 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:55.644 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:55.644 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:08:55.645 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:55.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:08:55.672 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:55.671+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:55.773 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:55.768+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:55.833 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:55.833 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:08:55.833 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:08:55.833 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:08:55.833 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:55.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:08:55.873 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:55.871+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:55.974 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:55.968+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807637114000002, 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:08:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807606264, 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:08:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807580096999999, 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:08:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:08:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807540622999994, 2024-09-13T09:08:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:08:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807506068999999, 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:08:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807473712999993, 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:08:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:08:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807441441999998, 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:08:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:08:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:08:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807407405999996, 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:08:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807374142, 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:08:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.807341902000005, 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:08:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:08:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:08:56.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.798361815999996, 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:08:56.083 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:08:56.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.087 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.798320935999996, 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.088 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.798280668000004, 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.093 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:08:56.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:08:56.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.798233624999995, 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:08:56.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:08:56.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:08:56.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.798188062999998, 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.102 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:08:56.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:08:56.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.798097894999998, 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.107 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:08:56.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:08:56.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.798051341000004, 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:08:56.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:08:56.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:08:56.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.116 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.796410417000004, 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.117 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:08:56.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.120 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:08:56.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.796344610999995, 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.126 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 23.113336965999999, 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.127 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:56.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:08:56.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.132 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.133 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:08:56.133 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:08:56.133 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:08:56.134 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:56.128+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:56.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:08:56.235 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:56.228+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:56.279 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:56.279 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:56.279 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:56.294 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:08:56.327 INFO:teuthology.orchestra.run.smithi031.stdout:4655 write 0x7d603cd thru 0x7d6dddc (0xda10 bytes) 2024-09-13T09:08:56.336 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:56.328+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:56.349 INFO:teuthology.orchestra.run.smithi031.stdout:4656 write 0xdb3a3a3 thru 0xdb482a0 (0xdefe bytes) 2024-09-13T09:08:56.365 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:56.365 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:56.365 INFO:teuthology.orchestra.run.smithi031.stdout:4659 read 0x5fa1ecd thru 0x5fa8986 (0x6aba bytes) 2024-09-13T09:08:56.365 INFO:teuthology.orchestra.run.smithi031.stdout:4660 write 0x1f6eb12 thru 0x1f7b500 (0xc9ef bytes) 2024-09-13T09:08:56.381 INFO:teuthology.orchestra.run.smithi031.stdout:4661 punch from 0x4bfbb4b to 0x4bfd8fc, (0x1db1 bytes) 2024-09-13T09:08:56.382 INFO:teuthology.orchestra.run.smithi031.stdout:4662 punch from 0xaefea66 to 0xaf08b90, (0xa12a bytes) 2024-09-13T09:08:56.382 INFO:teuthology.orchestra.run.smithi031.stdout:4663 read 0xc9517b9 thru 0xc9560af (0x48f7 bytes) 2024-09-13T09:08:56.382 INFO:teuthology.orchestra.run.smithi031.stdout:4664 read 0x85cf5de thru 0x85d0cd1 (0x16f4 bytes) 2024-09-13T09:08:56.382 INFO:teuthology.orchestra.run.smithi031.stdout:4665 write 0x44e2c79 thru 0x44ea559 (0x78e1 bytes) 2024-09-13T09:08:56.391 INFO:teuthology.orchestra.run.smithi031.stdout:4666 read 0x3be8803 thru 0x3bf1cf0 (0x94ee bytes) 2024-09-13T09:08:56.392 INFO:teuthology.orchestra.run.smithi031.stdout:4667 trunc from 0xe8806a1 to 0xdaa30cb 2024-09-13T09:08:56.424 INFO:teuthology.orchestra.run.smithi031.stdout:4668 trunc from 0xdaa30cb to 0x99ed3c 2024-09-13T09:08:56.436 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:56.434+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:56.455 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:56.455 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:56.455 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:56.461 INFO:teuthology.orchestra.run.smithi031.stdout:4669 write 0x445d438 thru 0x4468e38 (0xba01 bytes) 2024-09-13T09:08:56.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:08:56.475 INFO:teuthology.orchestra.run.smithi031.stdout:4670 punch from 0x2d5a22e to 0x2d66ec5, (0xcc97 bytes) 2024-09-13T09:08:56.497 INFO:teuthology.orchestra.run.smithi031.stdout:4671 write 0xd9830e1 thru 0xd991da6 (0xecc6 bytes) 2024-09-13T09:08:56.512 INFO:teuthology.orchestra.run.smithi031.stdout:4672 read 0x89736a2 thru 0x89781f0 (0x4b4f bytes) 2024-09-13T09:08:56.512 INFO:teuthology.orchestra.run.smithi031.stdout:4673 write 0x3dd9593 thru 0x3de83f1 (0xee5f bytes) 2024-09-13T09:08:56.527 INFO:teuthology.orchestra.run.smithi031.stdout:4674 writesame 0x6eee205 thru 0x6ef3401 data_size 0x1(0x51fd bytes) 2024-09-13T09:08:56.537 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:56.534+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:56.541 INFO:teuthology.orchestra.run.smithi031.stdout:4675 read 0xbf019c thru 0xbfe41c (0xe281 bytes) 2024-09-13T09:08:56.551 INFO:teuthology.orchestra.run.smithi031.stdout:4677 writesame 0xeafe0aa thru 0xeb084bf data_size 0x1(0xa416 bytes) 2024-09-13T09:08:56.551 INFO:teuthology.orchestra.run.smithi031.stdout:4678 write 0x3437d0 thru 0x3536cb (0xfefc bytes) 2024-09-13T09:08:56.557 INFO:teuthology.orchestra.run.smithi031.stdout:4679 read 0x92a451f thru 0x92a6b4c (0x262e bytes) 2024-09-13T09:08:56.557 INFO:teuthology.orchestra.run.smithi031.stdout:4680 write 0x2882923 thru 0x288eba6 (0xc284 bytes) 2024-09-13T09:08:56.604 INFO:teuthology.orchestra.run.smithi031.stdout:4681 read 0x580d17a thru 0x5819e65 (0xccec bytes) 2024-09-13T09:08:56.613 INFO:teuthology.orchestra.run.smithi031.stdout:4683 trunc from 0xeb084c0 to 0x95a0835 2024-09-13T09:08:56.636 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:56.636 INFO:teuthology.orchestra.run.smithi031.stdout:4686 read 0x7519f88 thru 0x752128c (0x7305 bytes) 2024-09-13T09:08:56.636 INFO:teuthology.orchestra.run.smithi031.stdout:4687 punch from 0x647003f to 0x6475e91, (0x5e52 bytes) 2024-09-13T09:08:56.636 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:56.638 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:56.634+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:56.639 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:56.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.525226959999998, 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.525206193000002, 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.645 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:08:56.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:08:56.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.525188341000003, 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:08:56.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:08:56.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.525171774999997, 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.655 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:08:56.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.525146190000001, 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:08:56.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.525127023000003, 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:56.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.525109739999998, 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:56.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:08:56.673 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.524580870000001, 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.674 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:08:56.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:08:56.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.678 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.524548965000001, 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:08:56.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.524524016999997, 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:08:56.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:08:56.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.524219451999997, 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:08:56.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.522457029000002, 2024-09-13T09:08:56.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:08:56.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.522436179000003, 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:08:56.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:08:56.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.522413855000003, 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:08:56.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.522395166000003, 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:08:56.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:08:56.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:08:56.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.522378113999999, 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:08:56.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.522361089999997, 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:08:56.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.512348103000001, 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:56.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:08:56.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.132193579999999, 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:08:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:08:56.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.132013157999999, 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:56.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:56.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:56.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:08:56.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:08:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:56.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:56.738 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:56.739 INFO:teuthology.orchestra.run.smithi031.stdout:4689 writesame 0x9f13b3f thru 0x9f1b61f data_size 0x1(0x7ae1 bytes) 2024-09-13T09:08:56.739 INFO:teuthology.orchestra.run.smithi031.stdout:4691 writesame 0x3279cea thru 0x327d080 data_size 0x1(0x3397 bytes) 2024-09-13T09:08:56.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4693 writesame 0xa71d2d0 thru 0xa72c87f data_size 0x8(0xf5b0 bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4696 punch from 0x2824058 to 0x28331ad, (0xf155 bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4697 writesame 0x19bb77c thru 0x19bf8f5 data_size 0x22(0x417a bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4698 read 0x2ff01f6 thru 0x2ff4af3 (0x48fe bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4699 read 0x97ab43c thru 0x97b65fe (0xb1c3 bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4701 write 0x166ea1a thru 0x167b6af (0xcc96 bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4702 write 0x231bf2a thru 0x2325f12 (0x9fe9 bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4703 read 0x3cb3e41 thru 0x3cbc8b6 (0x8a76 bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4704 punch from 0x7d2736f to 0x7d36339, (0xefca bytes) 2024-09-13T09:08:56.742 INFO:teuthology.orchestra.run.smithi031.stdout:4705 write 0x30b86de thru 0x30c3600 (0xaf23 bytes) 2024-09-13T09:08:56.743 INFO:teuthology.orchestra.run.smithi031.stdout:4706 clone 37 order 22 su 16384 sc 7 2024-09-13T09:08:56.743 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:56.732+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:56.840 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:56.838+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:56.907 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:56.908 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:56.908 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:56.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:08:56.941 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:56.936+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:56.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:08:57.041 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:57.038+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:57.086 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:08:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:08:57.087 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:57.103 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:08:57.143 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:57.136+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:57.243 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:57.242+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:57.274 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.319624628, 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:08:57.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:08:57.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:08:57.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88209 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24748032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.037075035999997, 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033308870999999997, 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88209 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:57.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235077+0000", 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.22e-06 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235080+0000", 2024-09-13T09:08:57.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:08:57.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235083+0000", 2024-09-13T09:08:57.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:08:57.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268290+0000", 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033207532999999997 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268342+0000", 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999997e-05 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268385+0000", 2024-09-13T09:08:57.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2874999999999999e-05 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88210 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24772608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.037066565000003, 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033438816000000003, 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88210 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235084+0000", 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235086+0000", 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235088+0000", 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235091+0000", 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4839999999999998e-06 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268421+0000", 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033329923999999997 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268479+0000", 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8529999999999997e-05 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268523+0000", 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4048999999999998e-05 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88211 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24797184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.037058100000003, 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033554247000000002, 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88211 2024-09-13T09:08:57.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235094+0000", 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2300000000000001e-06 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235097+0000", 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.021e-06 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235099+0000", 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268550+0000", 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033450510000000003 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268593+0000", 2024-09-13T09:08:57.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3035999999999999e-05 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268647+0000", 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4336999999999998e-05 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88212 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24821760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.037049711999998, 2024-09-13T09:08:57.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033675415, 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88212 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235102+0000", 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235105+0000", 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235108+0000", 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.367e-06 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268686+0000", 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033577810999999999 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268737+0000", 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1134999999999997e-05 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268777+0000", 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0187e-05 2024-09-13T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88213 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24846336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.037040509000001, 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033785911000000002, 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88213 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235110+0000", 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235112+0000", 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.728e-06 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235119+0000", 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:08:57.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235124+0000", 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268806+0000", 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033682479000000001 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268855+0000", 2024-09-13T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999998e-05 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268897+0000", 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1915000000000001e-05 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88214 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24870912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.037022874000002, 2024-09-13T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033897243000000001, 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88214 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235130+0000", 2024-09-13T09:08:57.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235139+0000", 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235151+0000", 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1856e-05 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268933+0000", 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033782874999999997 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268986+0000", 2024-09-13T09:08:57.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2593000000000003e-05 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269026+0000", 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9546999999999999e-05 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88215 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24895488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.310 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036997487000001, 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033999110999999999, 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88215 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235153+0000", 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235155+0000", 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8810000000000001e-06 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235159+0000", 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235161+0000", 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269056+0000", 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033895001000000001 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269109+0000", 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269153+0000", 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88216 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24920064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036987729000003, 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120667, 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88216 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235173+0000", 2024-09-13T09:08:57.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8490000000000001e-06 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235178+0000", 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235180+0000", 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269188+0000", 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034007280000000001 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269245+0000", 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6891000000000001e-05 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269284+0000", 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9561999999999998e-05 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.321 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88217 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24944640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036968315000003, 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034230180999999998, 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88217 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:08:57.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235184+0000", 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3120000000000001e-06 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235187+0000", 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235192+0000", 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269313+0000", 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120708 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269363+0000", 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9453999999999998e-05 2024-09-13T09:08:57.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269413+0000", 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0531000000000001e-05 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88218 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24969216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036956758000002, 2024-09-13T09:08:57.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034354619000000003, 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88218 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235195+0000", 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235199+0000", 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235201+0000", 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269450+0000", 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034248860999999999 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269502+0000", 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2519000000000003e-05 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269549+0000", 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6740999999999999e-05 2024-09-13T09:08:57.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88219 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24993792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036948217000003, 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034466610000000002, 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88219 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235204+0000", 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.184e-06 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235207+0000", 2024-09-13T09:08:57.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235209+0000", 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269577+0000", 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034368528000000002 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269628+0000", 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0241999999999998e-05 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269670+0000", 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1875999999999997e-05 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88220 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25018368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036940367, 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034591746999999999, 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88220 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:57.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235212+0000", 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0839999999999999e-06 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235215+0000", 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9830000000000001e-06 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235217+0000", 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269707+0000", 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034489989999999998 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269762+0000", 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000003e-05 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269803+0000", 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0181e-05 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88221 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25042944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036932284000002, 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034709489000000003, 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88221 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:57.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235220+0000", 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0640000000000001e-06 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235223+0000", 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235225+0000", 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269830+0000", 2024-09-13T09:08:57.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034604633000000003 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269884+0000", 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067000000000003e-05 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269928+0000", 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4607000000000002e-05 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.344 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88222 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25067520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036924138000003, 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034835487999999998, 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88222 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235228+0000", 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235231+0000", 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235233+0000", 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269963+0000", 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034730039999999997 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270016+0000", 2024-09-13T09:08:57.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2581999999999999e-05 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270063+0000", 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6718000000000002e-05 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88223 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25092096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036915917000002, 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034953250999999998, 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88223 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235236+0000", 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.127e-06 2024-09-13T09:08:57.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235239+0000", 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0180000000000002e-06 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235241+0000", 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270091+0000", 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034849036 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270137+0000", 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6112999999999997e-05 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270189+0000", 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88224 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25116672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036907718000002, 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035071738999999998, 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88224 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235244+0000", 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235247+0000", 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7750000000000001e-06 2024-09-13T09:08:57.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235249+0000", 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270226+0000", 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034977019999999998 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270275+0000", 2024-09-13T09:08:57.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8791000000000001e-05 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270315+0000", 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9953999999999997e-05 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88225 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25141248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:57.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.036899806000001, 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035182109000000003, 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88225 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:57.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235252+0000", 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235255+0000", 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235258+0000", 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270337+0000", 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035079359999999997 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270388+0000", 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0986000000000003e-05 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270433+0000", 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5581000000000003e-05 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93981 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [watch reconnect cookie 140193706156944 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.418052267999997, 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116345170000001, 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:08:57.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93981 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:08:57.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854096+0000", 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854113+0000", 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6759999999999999e-05 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854146+0000", 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3506000000000001e-05 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854169+0000", 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2331999999999999e-05 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856384+0000", 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022150960000000002 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856397+0000", 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3047999999999999e-05 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863781+0000", 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007384276 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863918+0000", 2024-09-13T09:08:57.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013628800000000001 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865733+0000", 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018159109999999999 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93982 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:08:57.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.414388336000002, 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0094988519999999, 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93982 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:08:57.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857761+0000", 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857770+0000", 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2109999999999998e-06 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857789+0000", 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9111999999999999e-05 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857800+0000", 2024-09-13T09:08:57.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857854+0000", 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4030999999999998e-05 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857866+0000", 2024-09-13T09:08:57.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2275e-05 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864650+0000", 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0067835570000001 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864768+0000", 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118019 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:08:57.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865240+0000", 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047203799999999998 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.866250+0000", 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001009935 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867165+0000", 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091516600000000003 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867219+0000", 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4330999999999999e-05 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867262+0000", 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.265e-05 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:08:57.377 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:57.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:08:57.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:08:57.380 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:57.378+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:57.480 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:57.476+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:57.548 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:08:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:08:57.549 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:08:57.549 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:08:57.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:08:57.570 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:57.570 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:57.581 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:08:57.576+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:08:57.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:57.587 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:57.603 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:57.604 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:57.621 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:57.622 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:57.642 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:57.642 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:57.662 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:57.662 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:08:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:57.682 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:57.678+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:57.683 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:08:57.684 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:08:57.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:08:57.704 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:08:57.705 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:08:57.783 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:57.782+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:57.836 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:57.836 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:57.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:08:57.891 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:57.890+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:57.954 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:08:57.985 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:57.982+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:58.007 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone36 from 0xa72c880 (overlap 0x3af40e) to 0xbca69 2024-09-13T09:08:58.036 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:58.036 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:58.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:08:58.087 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:58.080+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:58.137 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-13T09:08:58.188 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:58.186+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:58.221 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:58.221 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:58.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:08:58.327 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:58.326+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:08:58.367 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:58.367 INFO:teuthology.orchestra.run.smithi031.stdout:4708 writesame 0x89c8032 thru 0x89cc82c data_size 0x1(0x47fb bytes) 2024-09-13T09:08:58.389 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:58.384+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:58.402 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:58.402 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:58.405 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:58.405 INFO:teuthology.orchestra.run.smithi031.stdout:4710 read 0x727e89f thru 0x727feba (0x161c bytes) 2024-09-13T09:08:58.407 INFO:teuthology.orchestra.run.smithi031.stdout:4711 punch from 0x7b883df to 0x7b93017, (0xac38 bytes) 2024-09-13T09:08:58.407 INFO:teuthology.orchestra.run.smithi031.stdout:4713 write 0x14f51f thru 0x15506d (0x5b4f bytes) 2024-09-13T09:08:58.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:08:58.475 INFO:teuthology.orchestra.run.smithi031.stdout:4714 write 0x8e67eab thru 0x8e73df9 (0xbf4f bytes) 2024-09-13T09:08:58.490 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:58.486+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:58.494 INFO:teuthology.orchestra.run.smithi031.stdout:4715 write 0x705c5ea thru 0x7068323 (0xbd3a bytes) 2024-09-13T09:08:58.504 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:58.504 INFO:teuthology.orchestra.run.smithi031.stdout:4717 read 0x51a94c9 thru 0x51a9c02 (0x73a bytes) 2024-09-13T09:08:58.525 INFO:teuthology.orchestra.run.smithi031.stdout:4718 writesame 0xeb4f35a thru 0xeb5c597 data_size 0x11(0xd23e bytes) 2024-09-13T09:08:58.525 INFO:teuthology.orchestra.run.smithi031.stdout:4720 writesame 0x4315ba0 thru 0x431fc41 data_size 0x2(0xa0a2 bytes) 2024-09-13T09:08:58.575 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:58.575 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:58.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:08:58.592 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:58.584+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:58.595 INFO:teuthology.orchestra.run.smithi031.stdout:4721 write 0xdf3660d thru 0xdf3695b (0x34f bytes) 2024-09-13T09:08:58.600 INFO:teuthology.orchestra.run.smithi031.stdout:4722 read 0x1f8ad4b thru 0x1f95200 (0xa4b6 bytes) 2024-09-13T09:08:58.601 INFO:teuthology.orchestra.run.smithi031.stdout:4723 writesame 0xd342b87 thru 0xd3490bc data_size 0x5(0x6536 bytes) 2024-09-13T09:08:58.608 INFO:teuthology.orchestra.run.smithi031.stdout:4726 writesame 0x98cf86 thru 0x9965e7 data_size 0x1(0x9662 bytes) 2024-09-13T09:08:58.667 INFO:teuthology.orchestra.run.smithi031.stdout:4728 punch from 0xa22453a to 0xa22bf44, (0x7a0a bytes) 2024-09-13T09:08:58.667 INFO:teuthology.orchestra.run.smithi031.stdout:4729 write 0x60953fe thru 0x609ef41 (0x9b44 bytes) 2024-09-13T09:08:58.689 INFO:teuthology.orchestra.run.smithi031.stdout:4730 read 0x3b90be3 thru 0x3b9c4cf (0xb8ed bytes) 2024-09-13T09:08:58.690 INFO:teuthology.orchestra.run.smithi031.stdout:4731 write 0x246bf0f thru 0x2474170 (0x8262 bytes) 2024-09-13T09:08:58.692 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:58.690+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:58.787 INFO:teuthology.orchestra.run.smithi031.stdout:4733 write 0x90b8a7b thru 0x90bf500 (0x6a86 bytes) 2024-09-13T09:08:58.788 INFO:teuthology.orchestra.run.smithi031.stdout:4734 read 0x2abde7a thru 0x2ac11b6 (0x333d bytes) 2024-09-13T09:08:58.789 INFO:teuthology.orchestra.run.smithi031.stdout:4736 read 0xb90afc2 thru 0xb911592 (0x65d1 bytes) 2024-09-13T09:08:58.789 INFO:teuthology.orchestra.run.smithi031.stdout:4737 read 0xd757aba thru 0xd765a63 (0xdfaa bytes) 2024-09-13T09:08:58.790 INFO:teuthology.orchestra.run.smithi031.stdout:4738 writesame 0x65d2b0d thru 0x65d711b data_size 0x5(0x460f bytes) 2024-09-13T09:08:58.793 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:08:58.788+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:08:58.802 INFO:teuthology.orchestra.run.smithi031.stdout:4739 write 0x15b6e80 thru 0x15ba441 (0x35c2 bytes) 2024-09-13T09:08:58.804 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:58.804 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:58.804 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:58.804 INFO:teuthology.orchestra.run.smithi031.stdout:4744 write 0x41e765f thru 0x41e8c4e (0x15f0 bytes) 2024-09-13T09:08:58.807 INFO:teuthology.orchestra.run.smithi031.stdout:4745 writesame 0xd2c8abb thru 0xd2d48f7 data_size 0x1(0xbe3d bytes) 2024-09-13T09:08:58.809 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:58.809 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:58.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:08:58.851 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:58.851 INFO:teuthology.orchestra.run.smithi031.stdout:4747 read 0x4ac73fe thru 0x4ac8760 (0x1363 bytes) 2024-09-13T09:08:58.852 INFO:teuthology.orchestra.run.smithi031.stdout:4748 punch from 0x9977b97 to 0x9987a0c, (0xfe75 bytes) 2024-09-13T09:08:58.852 INFO:teuthology.orchestra.run.smithi031.stdout:4749 write 0x86ba1a1 thru 0x86c9e6f (0xfccf bytes) 2024-09-13T09:08:58.861 INFO:teuthology.orchestra.run.smithi031.stdout:4750 writesame 0xcb0f209 thru 0xcb18935 data_size 0xd(0x972d bytes) 2024-09-13T09:08:58.867 INFO:teuthology.orchestra.run.smithi031.stdout:4751 trunc from 0xeb5c598 to 0xeb36ffd 2024-09-13T09:08:58.881 INFO:teuthology.orchestra.run.smithi031.stdout:4753 read 0xaeb387b thru 0xaeb9103 (0x5889 bytes) 2024-09-13T09:08:58.881 INFO:teuthology.orchestra.run.smithi031.stdout:4754 punch from 0x26a62f1 to 0x26a742f, (0x113e bytes) 2024-09-13T09:08:58.881 INFO:teuthology.orchestra.run.smithi031.stdout:4755 writesame 0xea05cb0 thru 0xea0a45a data_size 0x1(0x47ab bytes) 2024-09-13T09:08:58.894 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:58.890+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:58.897 INFO:teuthology.orchestra.run.smithi031.stdout:4759 writesame 0xb894234 thru 0xb8979a6 data_size 0x5(0x3773 bytes) 2024-09-13T09:08:58.903 INFO:teuthology.orchestra.run.smithi031.stdout:4761 writesame 0x1e09ef0 thru 0x1e0bf3b data_size 0x35(0x204c bytes) 2024-09-13T09:08:58.923 INFO:teuthology.orchestra.run.smithi031.stdout:4762 write 0x8008caf thru 0x80097ef (0xb41 bytes) 2024-09-13T09:08:58.924 INFO:teuthology.orchestra.run.smithi031.stdout:4763 read 0x6d4c00 thru 0x6e3a9b (0xee9c bytes) 2024-09-13T09:08:58.925 INFO:teuthology.orchestra.run.smithi031.stdout:4764 write 0xab63c40 thru 0xab67567 (0x3928 bytes) 2024-09-13T09:08:58.929 INFO:teuthology.orchestra.run.smithi031.stdout:4766 read 0xb17587c thru 0xb17c0ac (0x6831 bytes) 2024-09-13T09:08:58.946 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 1] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:08:58.947 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:08:58.947 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 2, 3, 4, 5, 6, 1] 2024-09-13T09:08:58.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-13T09:08:58.952 INFO:teuthology.orchestra.run.smithi031.stdout:4767 trunc from 0xeb36ffd to 0x1fc1101 2024-09-13T09:08:58.995 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:58.988+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:59.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:08:59.044 INFO:teuthology.orchestra.run.smithi031.stdout:4771 write 0xac22319 thru 0xac2bf8e (0x9c76 bytes) 2024-09-13T09:08:59.050 INFO:teuthology.orchestra.run.smithi031.stdout:4772 punch from 0x30dc190 to 0x30e145d, (0x52cd bytes) 2024-09-13T09:08:59.059 INFO:teuthology.orchestra.run.smithi031.stdout:4773 write 0xc09af36 thru 0xc0a2d7e (0x7e49 bytes) 2024-09-13T09:08:59.064 INFO:teuthology.orchestra.run.smithi031.stdout:4775 write 0x1362db thru 0x142c94 (0xc9ba bytes) 2024-09-13T09:08:59.067 INFO:teuthology.orchestra.run.smithi031.stdout:4776 writesame 0x3536b42 thru 0x353fed8 data_size 0x1(0x9397 bytes) 2024-09-13T09:08:59.106 INFO:teuthology.orchestra.run.smithi031.stdout:4779 writesame 0xe175788 thru 0xe1772b6 data_size 0x1(0x1b2f bytes) 2024-09-13T09:08:59.106 INFO:teuthology.orchestra.run.smithi031.stdout:4780 read 0xd9ff554 thru 0xda03017 (0x3ac4 bytes) 2024-09-13T09:08:59.107 INFO:teuthology.orchestra.run.smithi031.stdout:4781 writesame 0x994fa74 thru 0x995771a data_size 0xb(0x7ca7 bytes) 2024-09-13T09:08:59.115 INFO:teuthology.orchestra.run.smithi031.stdout:4782 read 0xc81160c thru 0xc821064 (0xfa59 bytes) 2024-09-13T09:08:59.115 INFO:teuthology.orchestra.run.smithi031.stdout:4783 write 0x7803665 thru 0x780ed5a (0xb6f6 bytes) 2024-09-13T09:08:59.116 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:08:59.112+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:08:59.123 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.123 INFO:teuthology.orchestra.run.smithi031.stdout:4790 writesame 0x5b88ffc thru 0x5b98ed9 data_size 0x22(0xfede bytes) 2024-09-13T09:08:59.136 INFO:teuthology.orchestra.run.smithi031.stdout:4791 trunc from 0xe1772b7 to 0xb230b41 2024-09-13T09:08:59.146 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.146 INFO:teuthology.orchestra.run.smithi031.stdout:4794 write 0x7fad460 thru 0x7fbbdc7 (0xe968 bytes) 2024-09-13T09:08:59.164 INFO:teuthology.orchestra.run.smithi031.stdout:4797 writesame 0xe52d9ce thru 0xe53c6f7 data_size 0x9(0xed2a bytes) 2024-09-13T09:08:59.164 INFO:teuthology.orchestra.run.smithi031.stdout:4799 write 0x76859cc thru 0x7687bc4 (0x21f9 bytes) 2024-09-13T09:08:59.165 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.165 INFO:teuthology.orchestra.run.smithi031.stdout:4802 writesame 0xd1458f4 thru 0xd153309 data_size 0xf(0xda16 bytes) 2024-09-13T09:08:59.174 INFO:teuthology.orchestra.run.smithi031.stdout:4805 read 0xc0fe3 thru 0xcb4c9 (0xa4e7 bytes) 2024-09-13T09:08:59.185 INFO:teuthology.orchestra.run.smithi031.stdout:4806 trunc from 0xe53c6f8 to 0x8b98ccc 2024-09-13T09:08:59.197 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:59.194+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:59.209 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:59.209 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:59.219 INFO:teuthology.orchestra.run.smithi031.stdout:4807 trunc from 0x8b98ccc to 0x694098 2024-09-13T09:08:59.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:08:59.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.294 INFO:teuthology.orchestra.run.smithi031.stdout:4810 writesame 0x9fbf1a0 thru 0x9fc8f55 data_size 0x9(0x9db6 bytes) 2024-09-13T09:08:59.296 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:08:59.294+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:08:59.303 INFO:teuthology.orchestra.run.smithi031.stdout:4811 write 0xa7ba7f7 thru 0xa7c6406 (0xbc10 bytes) 2024-09-13T09:08:59.329 INFO:teuthology.orchestra.run.smithi031.stdout:4812 write 0xd414447 thru 0xd422852 (0xe40c bytes) 2024-09-13T09:08:59.339 INFO:teuthology.orchestra.run.smithi031.stdout:4813 write 0x4ecaa44 thru 0x4ed0edb (0x6498 bytes) 2024-09-13T09:08:59.348 INFO:teuthology.orchestra.run.smithi031.stdout:4814 writesame 0x204465f thru 0x2048e0b data_size 0x1(0x47ad bytes) 2024-09-13T09:08:59.350 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-13T09:08:59.346+0000 7f426cbbd640 -1 mon.a@0(leader).osd e104 definitely_dead 0 2024-09-13T09:08:59.371 INFO:teuthology.orchestra.run.smithi031.stdout:4815 write 0x7dcba69 thru 0x7dd0dac (0x5344 bytes) 2024-09-13T09:08:59.383 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.397 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:59.394+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:59.400 INFO:teuthology.orchestra.run.smithi031.stdout:4817 write 0xeb12a29 thru 0xeb19f44 (0x751c bytes) 2024-09-13T09:08:59.403 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:59.404 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:59.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:08:59.428 INFO:teuthology.orchestra.run.smithi031.stdout:4819 trunc from 0xeb19f45 to 0x85a231f 2024-09-13T09:08:59.457 INFO:teuthology.orchestra.run.smithi031.stdout:4820 write 0x5987765 thru 0x5994dc3 (0xd65f bytes) 2024-09-13T09:08:59.472 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.497 INFO:teuthology.orchestra.run.smithi031.stdout:4823 write 0xe81363a thru 0xe822064 (0xea2b bytes) 2024-09-13T09:08:59.498 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:59.494+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:59.510 INFO:teuthology.orchestra.run.smithi031.stdout:4825 read 0x8152b64 thru 0x8160ef2 (0xe38f bytes) 2024-09-13T09:08:59.511 INFO:teuthology.orchestra.run.smithi031.stdout:4826 write 0x92eea8f thru 0x92fa0be (0xb630 bytes) 2024-09-13T09:08:59.521 INFO:teuthology.orchestra.run.smithi031.stdout:4827 write 0xd20bf25 thru 0xd20d7b5 (0x1891 bytes) 2024-09-13T09:08:59.552 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:59.552 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:59.555 INFO:teuthology.orchestra.run.smithi031.stdout:4828 trunc from 0xe822065 to 0x5347bd2 2024-09-13T09:08:59.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:08:59.585 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.585 INFO:teuthology.orchestra.run.smithi031.stdout:4830 read 0x2984a37 thru 0x298849c (0x3a66 bytes) 2024-09-13T09:08:59.596 INFO:teuthology.orchestra.run.smithi031.stdout:4831 trunc from 0x5347bd2 to 0xbec0d5 2024-09-13T09:08:59.599 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:08:59.598+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:08:59.622 INFO:teuthology.orchestra.run.smithi031.stdout:4833 read 0x9c79dd thru 0x9d4dae (0xd3d2 bytes) 2024-09-13T09:08:59.623 INFO:teuthology.orchestra.run.smithi031.stdout:4834 read 0x694efe thru 0x6a02c6 (0xb3c9 bytes) 2024-09-13T09:08:59.624 INFO:teuthology.orchestra.run.smithi031.stdout:4836 read 0x92bf2d thru 0x9393bc (0xd490 bytes) 2024-09-13T09:08:59.658 INFO:teuthology.orchestra.run.smithi031.stdout:4837 trunc from 0xbec0d5 to 0xd39cd87 2024-09-13T09:08:59.672 INFO:teuthology.orchestra.run.smithi031.stdout:4838 read 0x22d91bd thru 0x22e7f15 (0xed59 bytes) 2024-09-13T09:08:59.672 INFO:teuthology.orchestra.run.smithi031.stdout:4839 writesame 0x82559d9 thru 0x8255d18 data_size 0x10(0x340 bytes) 2024-09-13T09:08:59.695 INFO:teuthology.orchestra.run.smithi031.stdout:4840 trunc from 0xd39cd87 to 0x79cb64b 2024-09-13T09:08:59.700 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:59.692+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:59.702 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:59.702 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:59.712 INFO:teuthology.orchestra.run.smithi031.stdout:4841 read 0x6fa1d0 thru 0x6ffa11 (0x5842 bytes) 2024-09-13T09:08:59.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:08:59.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:08:59.741 INFO:teuthology.orchestra.run.smithi031.stdout:4842 write 0xd713a45 thru 0xd71568e (0x1c4a bytes) 2024-09-13T09:08:59.748 INFO:teuthology.orchestra.run.smithi031.stdout:4843 read 0x43b85d2 thru 0x43bb79d (0x31cc bytes) 2024-09-13T09:08:59.749 INFO:teuthology.orchestra.run.smithi031.stdout:4844 read 0x93cc2ee thru 0x93d5b44 (0x9857 bytes) 2024-09-13T09:08:59.749 INFO:teuthology.orchestra.run.smithi031.stdout:4845 punch from 0xd51b91c to 0xd528534, (0xcc18 bytes) 2024-09-13T09:08:59.749 INFO:teuthology.orchestra.run.smithi031.stdout:4846 punch from 0x517084c to 0x517de18, (0xd5cc bytes) 2024-09-13T09:08:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:4849 trunc from 0xd71568f to 0x8a075e2 2024-09-13T09:08:59.788 INFO:teuthology.orchestra.run.smithi031.stdout:4850 trunc from 0x8a075e2 to 0x16b0b52 2024-09-13T09:08:59.800 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:08:59.798+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:08:59.808 INFO:teuthology.orchestra.run.smithi031.stdout:4851 punch from 0xd43d05 to 0xd4d75d, (0x9a58 bytes) 2024-09-13T09:08:59.808 INFO:teuthology.orchestra.run.smithi031.stdout:4852 punch from 0x13a35fd to 0x13afe56, (0xc859 bytes) 2024-09-13T09:08:59.823 INFO:teuthology.orchestra.run.smithi031.stdout:4853 trunc from 0x16b0b52 to 0xa804816 2024-09-13T09:08:59.833 INFO:teuthology.orchestra.run.smithi031.stdout:4854 read 0x4a87223 thru 0x4a9308b (0xbe69 bytes) 2024-09-13T09:08:59.833 INFO:teuthology.orchestra.run.smithi031.stdout:4855 write 0x74d140a thru 0x74d9ed9 (0x8ad0 bytes) 2024-09-13T09:08:59.840 INFO:teuthology.orchestra.run.smithi031.stdout:4858 write 0x435a4a8 thru 0x4360829 (0x6382 bytes) 2024-09-13T09:08:59.849 INFO:teuthology.orchestra.run.smithi031.stdout:4860 read 0x9e31d5d thru 0x9e3aa78 (0x8d1c bytes) 2024-09-13T09:08:59.849 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.849 INFO:teuthology.orchestra.run.smithi031.stdout:4862 punch from 0x12b14dc to 0x12bf4d3, (0xdff7 bytes) 2024-09-13T09:08:59.855 INFO:teuthology.orchestra.run.smithi031.stdout:4863 trunc from 0xa804816 to 0x7c1c725 2024-09-13T09:08:59.867 INFO:teuthology.orchestra.run.smithi031.stdout:4864 read 0x6da245e thru 0x6dab877 (0x941a bytes) 2024-09-13T09:08:59.878 INFO:teuthology.orchestra.run.smithi031.stdout:4865 write 0x9b15c31 thru 0x9b21c02 (0xbfd2 bytes) 2024-09-13T09:08:59.892 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:08:59.892 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:08:59.896 INFO:teuthology.orchestra.run.smithi031.stdout:4866 write 0xaa7a3a7 thru 0xaa7e0b3 (0x3d0d bytes) 2024-09-13T09:08:59.897 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:08:59.897 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:08:59.900 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:08:59.896+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:08:59.902 INFO:teuthology.orchestra.run.smithi031.stdout:4868 writesame 0x65f07eb thru 0x65f2360 data_size 0x26(0x1b76 bytes) 2024-09-13T09:08:59.908 INFO:teuthology.orchestra.run.smithi031.stdout:4869 trunc from 0xaa7e0b4 to 0xc016b51 2024-09-13T09:08:59.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:08:59.910 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:08:59.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:08:59.918 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.918 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:08:59.919 INFO:teuthology.orchestra.run.smithi031.stdout:4874 write 0xbf4105 thru 0xc0152e (0xd42a bytes) 2024-09-13T09:08:59.928 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:08:59.928 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:08:59.928 INFO:teuthology.orchestra.run.smithi031.stdout:4875 trunc from 0xc016b51 to 0x77bc6a1 2024-09-13T09:08:59.945 INFO:teuthology.orchestra.run.smithi031.stdout:4876 punch from 0x170905 to 0x17df2f, (0xd62a bytes) 2024-09-13T09:08:59.945 INFO:teuthology.orchestra.run.smithi031.stdout:4877 write 0x26b5b99 thru 0x26c1c3a (0xc0a2 bytes) 2024-09-13T09:08:59.946 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:08:59.946 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:08:59.953 INFO:teuthology.orchestra.run.smithi031.stdout:4879 read 0x432cce thru 0x43f33b (0xc66e bytes) 2024-09-13T09:08:59.954 INFO:teuthology.orchestra.run.smithi031.stdout:4880 read 0x37b6031 thru 0x37bba34 (0x5a04 bytes) 2024-09-13T09:08:59.954 INFO:teuthology.orchestra.run.smithi031.stdout:4882 clone 38 order 23 su 2097152 sc 4 2024-09-13T09:08:59.961 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.0. 2024-09-13T09:08:59.969 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:08:59.969 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:08:59.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:08:59.988 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:08:59.988 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:00.001 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:08:59.998+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:00.007 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:00.007 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:00.025 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:00.025 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649411587000003, 2024-09-13T09:09:00.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:00.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:09:00.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649387004000005, 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:09:00.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649362999999994, 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:09:00.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:09:00.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649339517000001, 2024-09-13T09:09:00.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:00.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:09:00.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649316110000001, 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:09:00.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649292861000006, 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:09:00.117 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:09:00.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.120 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.121 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649260755, 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:09:00.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:09:00.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649225138000006, 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649193158000003, 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.131 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.133 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:09:00.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:09:00.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:09:00.136 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.136 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.136 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.137 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.137 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649123879000001, 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.138 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.142 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:09:00.142 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:09:00.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.142 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.142 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.142 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:09:00.142 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649098819000002, 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.143 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:00.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:09:00.147 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649075389000004, 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.148 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:09:00.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:09:00.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649052006000005, 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.153 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:00.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:09:00.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.157 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.64902859, 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:00.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.649004667, 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.162 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:00.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:09:00.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.648971583000005, 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.167 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.171 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.648947782999997, 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:09:00.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.648923456999995, 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.176 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:09:00.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.181 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.648890897000001, 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:09:00.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:09:00.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 84.648851023999995, 2024-09-13T09:09:00.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:09:00.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:09:00.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:09:00.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.191 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.191 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.191 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:00.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:00.193 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:00.184+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:00.293 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:00.290+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:00.365 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:00.365 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:00.365 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:00.366 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:00.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:00.394 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:00.390+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:00.497 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:00.492+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:00.509 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:00.509 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:00.510 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:00.510 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:00.510 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:00.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:00.598 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:00.594+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:00.686 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:00.686 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.460174280999993, 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.687 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:09:00.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.692 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.460143431000006, 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:09:00.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:09:00.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.697 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.460117264000004, 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.698 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:09:00.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:09:00.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:00.703 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.46007779, 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:09:00.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.460043236000004, 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:09:00.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:09:00.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:09:00.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.460010879999999, 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:09:00.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.459978609000004, 2024-09-13T09:09:00.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.459944573000001, 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.725 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:00.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:09:00.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.459911309000006, 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:09:00.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:09:00.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.735 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.459879068999996, 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:09:00.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:09:00.740 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.450898983000002, 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.741 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:09:00.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:09:00.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:09:00.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:09:00.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.746 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.450858103000002, 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:09:00.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:09:00.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.450817834999995, 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.752 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:09:00.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.757 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.450770792, 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:00.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.762 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.450725230000003, 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.763 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:09:00.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.450635062000003, 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.768 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:09:00.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:00.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.450588507999996, 2024-09-13T09:09:00.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:00.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:09:00.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.778 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.448947583999995, 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:09:00.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:09:00.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:09:00.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:09:00.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.448881778000001, 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:09:00.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:09:00.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.765874133000001, 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:00.790 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:00.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:00.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:09:00.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:00.796 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:00.797 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:00.798 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:00.796+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:00.798 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:00.899 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:00.898+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:00.926 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:00.926 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:00.926 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:00.926 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:00.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:09:01.043 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:01.042+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:01.089 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:01.089 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:01.089 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:01.089 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:01.089 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:01.100 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:01.098+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:01.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:01.201 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:01.198+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:01.267 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:01.267 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:01.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151837813999997, 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151817047000002, 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151799195000002, 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:09:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:09:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151782629000003, 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:09:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:09:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151757044, 2024-09-13T09:09:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:09:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:09:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151737877000002, 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151720593999997, 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:09:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151191724, 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:09:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151159819, 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:09:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:09:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:09:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.151134870999996, 2024-09-13T09:09:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:09:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:09:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.150830306000003, 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:09:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:09:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.149067883000001, 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:09:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.149047033000002, 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:09:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.149024709000003, 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:09:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:09:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.149006020000002, 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:09:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:09:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.148988967999998, 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.148971944000003, 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:09:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:09:01.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.138958957, 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.758804433999998, 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:09:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:09:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.758624011999999, 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.363 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:01.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:01.365 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:01.362+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:01.466 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:01.460+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:01.537 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:01.537 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:01.537 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:01.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:01.567 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:01.560+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:01.668 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:01.666+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:01.672 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:01.672 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:01.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:01.769 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:01.764+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 68.906410643000001, 2024-09-13T09:09:01.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88209 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24748032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:01.867 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623861050999999, 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033308870999999997, 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88209 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235077+0000", 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.22e-06 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235080+0000", 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235083+0000", 2024-09-13T09:09:01.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268290+0000", 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033207532999999997 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268342+0000", 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999997e-05 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268385+0000", 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2874999999999999e-05 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88210 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24772608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.872 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623852579999998, 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033438816000000003, 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88210 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235084+0000", 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235086+0000", 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235088+0000", 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235091+0000", 2024-09-13T09:09:01.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4839999999999998e-06 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268421+0000", 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033329923999999997 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268479+0000", 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8529999999999997e-05 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268523+0000", 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4048999999999998e-05 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88211 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24797184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623844114999997, 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033554247000000002, 2024-09-13T09:09:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88211 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235094+0000", 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2300000000000001e-06 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235097+0000", 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.021e-06 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235099+0000", 2024-09-13T09:09:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268550+0000", 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033450510000000003 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268593+0000", 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3035999999999999e-05 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268647+0000", 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4336999999999998e-05 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88212 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24821760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623835726999999, 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033675415, 2024-09-13T09:09:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88212 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235102+0000", 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235105+0000", 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235108+0000", 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.367e-06 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268686+0000", 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033577810999999999 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268737+0000", 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1134999999999997e-05 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268777+0000", 2024-09-13T09:09:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0187e-05 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88213 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24846336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623826524000002, 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033785911000000002, 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88213 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235110+0000", 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235112+0000", 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.728e-06 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235119+0000", 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235124+0000", 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268806+0000", 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033682479000000001 2024-09-13T09:09:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268855+0000", 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999998e-05 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268897+0000", 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1915000000000001e-05 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88214 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24870912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623808889000003, 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033897243000000001, 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88214 2024-09-13T09:09:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235130+0000", 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235139+0000", 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235151+0000", 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1856e-05 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268933+0000", 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033782874999999997 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268986+0000", 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2593000000000003e-05 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269026+0000", 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9546999999999999e-05 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88215 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24895488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623783502000002, 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033999110999999999, 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88215 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235153+0000", 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235155+0000", 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8810000000000001e-06 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235159+0000", 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235161+0000", 2024-09-13T09:09:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269056+0000", 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033895001000000001 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269109+0000", 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269153+0000", 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88216 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24920064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623773743999998, 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120667, 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88216 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235173+0000", 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8490000000000001e-06 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235178+0000", 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235180+0000", 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-13T09:09:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269188+0000", 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034007280000000001 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269245+0000", 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6891000000000001e-05 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269284+0000", 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9561999999999998e-05 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88217 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24944640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623754329999997, 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034230180999999998, 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88217 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235184+0000", 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3120000000000001e-06 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235187+0000", 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235192+0000", 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269313+0000", 2024-09-13T09:09:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120708 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269363+0000", 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9453999999999998e-05 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269413+0000", 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0531000000000001e-05 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88218 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24969216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623742773000004, 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034354619000000003, 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88218 2024-09-13T09:09:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235195+0000", 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235199+0000", 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:09:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235201+0000", 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269450+0000", 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034248860999999999 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269502+0000", 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2519000000000003e-05 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269549+0000", 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6740999999999999e-05 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88219 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24993792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623734231999997, 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034466610000000002, 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88219 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235204+0000", 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.184e-06 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235207+0000", 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-13T09:09:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235209+0000", 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269577+0000", 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034368528000000002 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269628+0000", 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0241999999999998e-05 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269670+0000", 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1875999999999997e-05 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88220 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25018368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623726382000001, 2024-09-13T09:09:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034591746999999999, 2024-09-13T09:09:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88220 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235212+0000", 2024-09-13T09:09:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0839999999999999e-06 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235215+0000", 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9830000000000001e-06 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235217+0000", 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269707+0000", 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034489989999999998 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269762+0000", 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000003e-05 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269803+0000", 2024-09-13T09:09:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0181e-05 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88221 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25042944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623718298999997, 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034709489000000003, 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88221 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235220+0000", 2024-09-13T09:09:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0640000000000001e-06 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235223+0000", 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235225+0000", 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269830+0000", 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034604633000000003 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269884+0000", 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067000000000003e-05 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269928+0000", 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4607000000000002e-05 2024-09-13T09:09:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88222 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25067520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623710152999998, 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034835487999999998, 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88222 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235228+0000", 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235231+0000", 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235233+0000", 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269963+0000", 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034730039999999997 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270016+0000", 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2581999999999999e-05 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270063+0000", 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6718000000000002e-05 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88223 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25092096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623701932000003, 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034953250999999998, 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88223 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235236+0000", 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.127e-06 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235239+0000", 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0180000000000002e-06 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235241+0000", 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270091+0000", 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034849036 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270137+0000", 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6112999999999997e-05 2024-09-13T09:09:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270189+0000", 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88224 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25116672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623693733000003, 2024-09-13T09:09:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035071738999999998, 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88224 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235244+0000", 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235247+0000", 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7750000000000001e-06 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235249+0000", 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270226+0000", 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034977019999999998 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270275+0000", 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8791000000000001e-05 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270315+0000", 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9953999999999997e-05 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88225 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25141248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.623685821000002, 2024-09-13T09:09:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035182109000000003, 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88225 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235252+0000", 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235255+0000", 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235258+0000", 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270337+0000", 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035079359999999997 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270388+0000", 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0986000000000003e-05 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270433+0000", 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5581000000000003e-05 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93981 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [watch reconnect cookie 140193706156944 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.004838282999998, 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116345170000001, 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93981 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854096+0000", 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854113+0000", 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6759999999999999e-05 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854146+0000", 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3506000000000001e-05 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854169+0000", 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2331999999999999e-05 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856384+0000", 2024-09-13T09:09:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022150960000000002 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856397+0000", 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3047999999999999e-05 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863781+0000", 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007384276 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863918+0000", 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013628800000000001 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865733+0000", 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018159109999999999 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93982 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.001174351000003, 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0094988519999999, 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93982 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857761+0000", 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857770+0000", 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2109999999999998e-06 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857789+0000", 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9111999999999999e-05 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857800+0000", 2024-09-13T09:09:01.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857854+0000", 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4030999999999998e-05 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857866+0000", 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2275e-05 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:01.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864650+0000", 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0067835570000001 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864768+0000", 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118019 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865240+0000", 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047203799999999998 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.866250+0000", 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001009935 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867165+0000", 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091516600000000003 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867219+0000", 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4330999999999999e-05 2024-09-13T09:09:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867262+0000", 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.265e-05 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:01.965 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:01.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:01.967 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:01.962+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:02.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:02.068 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:02.064+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:02.167 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "MOSDPGPush(3.13 107/106 [PushOp(3:ceb65316:::rbd_data.112498fc7c8e.0000000000000076:head, version: 89'345, data_included: [696320~4096], data_size: 4096, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:ceb65316:::rbd_data.112498fc7c8e.0000000000000076:head@89'345, size: 700416, copy_subset: [0~700416], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:700416, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-13T09:09:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:02.166918+0000", 2024-09-13T09:09:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.000154384, 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00019776700000000001, 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:02.166918+0000", 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:02.166918+0000", 2024-09-13T09:09:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:02.166917+0000", 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:02.166947+0000", 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9728999999999999e-05 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:02.166966+0000", 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9343000000000001e-05 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:02.166979+0000", 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3237999999999999e-05 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:02.167052+0000", 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2329999999999994e-05 2024-09-13T09:09:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:02.167063+0000", 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1439e-05 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-13T09:09:02.171 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:02.172 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:02.168+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:02.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:02.212 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:02.212 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:02.236 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:02.237 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:02.259 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:02.259 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:02.273 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:02.270+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:02.281 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:02.281 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:02.283 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone37 from 0x77bc6a1 (overlap 0x694098) to 0x3f26c1 2024-09-13T09:09:02.306 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:02.306 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:02.334 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:02.334 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:02.334 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:02.335 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:02.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:02.358 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:02.359 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:02.374 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:02.368+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:02.384 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:02.385 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:02.420 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-13T09:09:02.426 INFO:teuthology.orchestra.run.smithi031.stdout:4883 writesame 0x5927bf1 thru 0x592bb8e data_size 0x11(0x3f9e bytes) 2024-09-13T09:09:02.452 INFO:teuthology.orchestra.run.smithi031.stdout:4884 write 0x9c768af thru 0x9c8606b (0xf7bd bytes) 2024-09-13T09:09:02.458 INFO:teuthology.orchestra.run.smithi031.stdout:4886 punch from 0x50d0028 to 0x50d4786, (0x475e bytes) 2024-09-13T09:09:02.458 INFO:teuthology.orchestra.run.smithi031.stdout:4887 read 0x65fee77 thru 0x65ff656 (0x7e0 bytes) 2024-09-13T09:09:02.458 INFO:teuthology.orchestra.run.smithi031.stdout:4888 punch from 0x9671769 to 0x9676892, (0x5129 bytes) 2024-09-13T09:09:02.458 INFO:teuthology.orchestra.run.smithi031.stdout:4889 write 0x9263750 thru 0x926afa9 (0x785a bytes) 2024-09-13T09:09:02.462 INFO:teuthology.orchestra.run.smithi031.stdout:4890 write 0x7b0b1d2 thru 0x7b0f22c (0x405b bytes) 2024-09-13T09:09:02.469 INFO:teuthology.orchestra.run.smithi031.stdout:4891 write 0x18d9261 thru 0x18e6a9b (0xd83b bytes) 2024-09-13T09:09:02.475 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:02.468+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:02.489 INFO:teuthology.orchestra.run.smithi031.stdout:4894 read 0x1568331 thru 0x156bf66 (0x3c36 bytes) 2024-09-13T09:09:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:02.493 INFO:teuthology.orchestra.run.smithi031.stdout:4895 trunc from 0x9c8606c to 0x7cec148 2024-09-13T09:09:02.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:02.518 INFO:teuthology.orchestra.run.smithi031.stdout:4897 read 0x29a1aca thru 0x29aff29 (0xe460 bytes) 2024-09-13T09:09:02.520 INFO:teuthology.orchestra.run.smithi031.stdout:4898 punch from 0x75c5871 to 0x75c82c1, (0x2a50 bytes) 2024-09-13T09:09:02.520 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:02.520 INFO:teuthology.orchestra.run.smithi031.stdout:4901 read 0x3f29e4 thru 0x3f4297 (0x18b4 bytes) 2024-09-13T09:09:02.521 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:02.532 INFO:teuthology.orchestra.run.smithi031.stdout:4906 trunc from 0x7cec148 to 0x20b12e 2024-09-13T09:09:02.561 INFO:teuthology.orchestra.run.smithi031.stdout:4907 read 0x8bd25 thru 0x95bd3 (0x9eaf bytes) 2024-09-13T09:09:02.561 INFO:teuthology.orchestra.run.smithi031.stdout:4909 punch from 0x194862 to 0x1a20c4, (0xd862 bytes) 2024-09-13T09:09:02.561 INFO:teuthology.orchestra.run.smithi031.stdout:4910 punch from 0xc995f to 0xce206, (0x48a7 bytes) 2024-09-13T09:09:02.573 INFO:teuthology.orchestra.run.smithi031.stdout:4911 trunc from 0x20b12e to 0x8b4394f 2024-09-13T09:09:02.576 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:02.572+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:02.593 INFO:teuthology.orchestra.run.smithi031.stdout:4913 writesame 0xc9aa194 thru 0xc9b97d0 data_size 0xd(0xf63d bytes) 2024-09-13T09:09:02.603 INFO:teuthology.orchestra.run.smithi031.stdout:4914 trunc from 0xc9b97d1 to 0x53acbf7 2024-09-13T09:09:02.611 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:02.611 INFO:teuthology.orchestra.run.smithi031.stdout:4918 writesame 0x4f18136 thru 0x4f1df29 data_size 0xe(0x5df4 bytes) 2024-09-13T09:09:02.623 INFO:teuthology.orchestra.run.smithi031.stdout:4919 trunc from 0x53acbf7 to 0x8b92afe 2024-09-13T09:09:02.631 INFO:teuthology.orchestra.run.smithi031.stdout:4920 clone 39 order 25 su 16777216 sc 15 2024-09-13T09:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:02.648 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:02.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:02.677 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:02.676+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:02.778 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:02.774+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:02.830 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:02.831 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:02.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:02.880 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:02.876+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:02.980 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:02.972+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:03.000 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:09:03.022 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:03.022 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:03.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:03.085 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:03.080+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:03.173 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:03.173 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:03.181 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:03.178+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:03.187 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:03.282 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:03.276+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:03.348 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:03.348 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:03.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:03.383 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:03.382+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:03.484 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:03.476+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:03.526 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:03.527 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:03.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:03.586 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:03.582+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:03.686 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:03.682+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:03.705 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:03.705 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:03.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:03.787 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:03.780+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:03.847 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:03.848 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:03.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:03.905 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:03.900+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:03.990 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:03.986+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:04.004 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:09:04.034 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:04.034 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:04.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:04.067 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone38 intact 2024-09-13T09:09:04.090 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:04.086+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:04.175 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:04.175 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:04.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:04.191 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:04.190+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:04.293 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:04.288+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:04.302 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-13T09:09:04.317 INFO:teuthology.orchestra.run.smithi031.stdout:4921 write 0x7dc1328 thru 0x7dc6ed6 (0x5baf bytes) 2024-09-13T09:09:04.319 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:04.319 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:04.332 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.332 INFO:teuthology.orchestra.run.smithi031.stdout:4923 write 0x84c0629 thru 0x84c4106 (0x3ade bytes) 2024-09-13T09:09:04.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:04.339 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.339 INFO:teuthology.orchestra.run.smithi031.stdout:4925 read 0x689c3c5 thru 0x689e39b (0x1fd7 bytes) 2024-09-13T09:09:04.339 INFO:teuthology.orchestra.run.smithi031.stdout:4926 write 0xbf2fb7 thru 0xbfaf5e (0x7fa8 bytes) 2024-09-13T09:09:04.364 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.364 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.365 INFO:teuthology.orchestra.run.smithi031.stdout:4931 punch from 0x2fe4f59 to 0x2ff0c45, (0xbcec bytes) 2024-09-13T09:09:04.369 INFO:teuthology.orchestra.run.smithi031.stdout:4933 trunc from 0x8b92afe to 0xa797528 2024-09-13T09:09:04.397 INFO:teuthology.orchestra.run.smithi031.stdout:4934 write 0xac3f1af thru 0xac44f6b (0x5dbd bytes) 2024-09-13T09:09:04.404 INFO:teuthology.orchestra.run.smithi031.stdout:4935 write 0x7c41c7 thru 0x7d13b2 (0xd1ec bytes) 2024-09-13T09:09:04.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:04.411 INFO:teuthology.orchestra.run.smithi031.stdout:4936 trunc from 0xac44f6c to 0xc7ae9af 2024-09-13T09:09:04.421 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:04.416+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:04.425 INFO:teuthology.orchestra.run.smithi031.stdout:4938 trunc from 0xc7ae9af to 0xd101b87 2024-09-13T09:09:04.435 INFO:teuthology.orchestra.run.smithi031.stdout:4939 read 0x1b696e0 thru 0x1b79091 (0xf9b2 bytes) 2024-09-13T09:09:04.436 INFO:teuthology.orchestra.run.smithi031.stdout:4940 flatten 2024-09-13T09:09:04.455 INFO:teuthology.orchestra.run.smithi031.stdout:4941 read 0x4c9fba9 thru 0x4caa861 (0xacb9 bytes) 2024-09-13T09:09:04.456 INFO:teuthology.orchestra.run.smithi031.stdout:4944 write 0x4032c0b thru 0x40393be (0x67b4 bytes) 2024-09-13T09:09:04.458 INFO:teuthology.orchestra.run.smithi031.stdout:4945 writesame 0x3c6bcbf thru 0x3c7407a data_size 0x2(0x83bc bytes) 2024-09-13T09:09:04.481 INFO:teuthology.orchestra.run.smithi031.stdout:4948 writesame 0x52e74c2 thru 0x52eeee4 data_size 0x1(0x7a23 bytes) 2024-09-13T09:09:04.494 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:04.490+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:04.506 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:04.507 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:04.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:04.522 INFO:teuthology.orchestra.run.smithi031.stdout:4950 read 0x2c320db thru 0x2c3d362 (0xb288 bytes) 2024-09-13T09:09:04.523 INFO:teuthology.orchestra.run.smithi031.stdout:4951 writesame 0x99dbb3 thru 0x9a84a1 data_size 0x1(0xa8ef bytes) 2024-09-13T09:09:04.571 INFO:teuthology.orchestra.run.smithi031.stdout:4953 write 0x52cfbc7 thru 0x52dacc2 (0xb0fc bytes) 2024-09-13T09:09:04.575 INFO:teuthology.orchestra.run.smithi031.stdout:4954 read 0xc3a3452 thru 0xc3aa246 (0x6df5 bytes) 2024-09-13T09:09:04.575 INFO:teuthology.orchestra.run.smithi031.stdout:4955 read 0x958be34 thru 0x95932be (0x748b bytes) 2024-09-13T09:09:04.575 INFO:teuthology.orchestra.run.smithi031.stdout:4956 write 0xc9a75d6 thru 0xc9b588d (0xe2b8 bytes) 2024-09-13T09:09:04.575 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:04.576 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:04.579 INFO:teuthology.orchestra.run.smithi031.stdout:4957 read 0x91f49ab thru 0x91ffe99 (0xb4ef bytes) 2024-09-13T09:09:04.579 INFO:teuthology.orchestra.run.smithi031.stdout:4960 read 0xd07aea thru 0xd0ad13 (0x322a bytes) 2024-09-13T09:09:04.581 INFO:teuthology.orchestra.run.smithi031.stdout:4961 read 0x61f93a6 thru 0x6206fe0 (0xdc3b bytes) 2024-09-13T09:09:04.581 INFO:teuthology.orchestra.run.smithi031.stdout:4962 write 0x171972b thru 0x1721482 (0x7d58 bytes) 2024-09-13T09:09:04.585 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.585 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.585 INFO:teuthology.orchestra.run.smithi031.stdout:4965 writesame 0x1845a5f thru 0x18536d0 data_size 0x8b(0xdc72 bytes) 2024-09-13T09:09:04.587 INFO:teuthology.orchestra.run.smithi031.stdout:4966 trunc from 0xd101b87 to 0xc6546a0 2024-09-13T09:09:04.595 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:04.590+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:04.596 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:04.596 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:04.601 INFO:teuthology.orchestra.run.smithi031.stdout:4967 write 0x8be8f2a thru 0x8bf5972 (0xca49 bytes) 2024-09-13T09:09:04.603 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.603 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.607 INFO:teuthology.orchestra.run.smithi031.stdout:4972 trunc from 0xc6546a0 to 0x9d3ae74 2024-09-13T09:09:04.613 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:04.614 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:04.624 INFO:teuthology.orchestra.run.smithi031.stdout:4973 read 0x2dd9d07 thru 0x2de9b96 (0xfe90 bytes) 2024-09-13T09:09:04.624 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.624 INFO:teuthology.orchestra.run.smithi031.stdout:4975 punch from 0x53f7711 to 0x5405dcc, (0xe6bb bytes) 2024-09-13T09:09:04.624 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.631 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:04.631 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:04.635 INFO:teuthology.orchestra.run.smithi031.stdout:4977 write 0xc315c35 thru 0xc31ad76 (0x5142 bytes) 2024-09-13T09:09:04.650 INFO:teuthology.orchestra.run.smithi031.stdout:4978 write 0xd31b477 thru 0xd323408 (0x7f92 bytes) 2024-09-13T09:09:04.652 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:04.652 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:04.663 INFO:teuthology.orchestra.run.smithi031.stdout:4979 trunc from 0xd323409 to 0x7788320 2024-09-13T09:09:04.674 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:04.674 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:04.696 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:04.694+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:04.696 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:04.697 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:04.702 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:04.702 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:04.703 INFO:teuthology.orchestra.run.smithi031.stdout:4982 write 0xd76bfe6 thru 0xd777eb0 (0xbecb bytes) 2024-09-13T09:09:04.708 INFO:teuthology.orchestra.run.smithi031.stdout:4983 write 0x6f3a52 thru 0x6f7c60 (0x420f bytes) 2024-09-13T09:09:04.712 INFO:teuthology.orchestra.run.smithi031.stdout:4985 write 0x3854736 thru 0x385f757 (0xb022 bytes) 2024-09-13T09:09:04.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:04.714 INFO:teuthology.orchestra.run.smithi031.stdout:4986 read 0x764474e thru 0x7653bd3 (0xf486 bytes) 2024-09-13T09:09:04.724 INFO:teuthology.orchestra.run.smithi031.stdout:4988 trunc from 0xd777eb1 to 0x7a1304b 2024-09-13T09:09:04.739 INFO:teuthology.orchestra.run.smithi031.stdout:4989 write 0x3edd421 thru 0x3ee1df9 (0x49d9 bytes) 2024-09-13T09:09:04.745 INFO:teuthology.orchestra.run.smithi031.stdout:4992 trunc from 0x7a1304b to 0x98dd52f 2024-09-13T09:09:04.756 INFO:teuthology.orchestra.run.smithi031.stdout:4993 write 0x8d259e1 thru 0x8d2a9e9 (0x5009 bytes) 2024-09-13T09:09:04.761 INFO:teuthology.orchestra.run.smithi031.stdout:4995 write 0x802174a thru 0x8029af1 (0x83a8 bytes) 2024-09-13T09:09:04.763 INFO:teuthology.orchestra.run.smithi031.stdout:4996 write 0x6a6cc1 thru 0x6ae57f (0x78bf bytes) 2024-09-13T09:09:04.776 INFO:teuthology.orchestra.run.smithi031.stdout:4997 trunc from 0x98dd52f to 0xed2698c 2024-09-13T09:09:04.787 INFO:teuthology.orchestra.run.smithi031.stdout:4998 read 0x2d02283 thru 0x2d093fc (0x717a bytes) 2024-09-13T09:09:04.787 INFO:teuthology.orchestra.run.smithi031.stdout:4999 punch from 0x48c5498 to 0x48cb84c, (0x63b4 bytes) 2024-09-13T09:09:04.787 INFO:teuthology.orchestra.run.smithi031.stdout:5000 read 0x8d1fda9 thru 0x8d2ae49 (0xb0a1 bytes) 2024-09-13T09:09:04.788 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.789 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.789 INFO:teuthology.orchestra.run.smithi031.stdout:5003 read 0xd58c0e8 thru 0xd59901a (0xcf33 bytes) 2024-09-13T09:09:04.789 INFO:teuthology.orchestra.run.smithi031.stdout:5004 writesame 0x6b0a56e thru 0x6b14a0d data_size 0x3(0xa4a0 bytes) 2024-09-13T09:09:04.797 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:04.792+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:04.810 INFO:teuthology.orchestra.run.smithi031.stdout:5005 write 0xe2d6fb7 thru 0xe2e4f2a (0xdf74 bytes) 2024-09-13T09:09:04.818 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.818 INFO:teuthology.orchestra.run.smithi031.stdout:5008 write 0x213b196 thru 0x2145f6c (0xadd7 bytes) 2024-09-13T09:09:04.826 INFO:teuthology.orchestra.run.smithi031.stdout:5009 punch from 0x1bff541 to 0x1c040ba, (0x4b79 bytes) 2024-09-13T09:09:04.826 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:04.826 INFO:teuthology.orchestra.run.smithi031.stdout:5012 write 0x7a12dda thru 0x7a1ffcc (0xd1f3 bytes) 2024-09-13T09:09:04.830 INFO:teuthology.orchestra.run.smithi031.stdout:5013 punch from 0xa1fc70d to 0xa1feff8, (0x28eb bytes) 2024-09-13T09:09:04.830 INFO:teuthology.orchestra.run.smithi031.stdout:5015 clone 40 order 19 su 262144 sc 11 2024-09-13T09:09:04.844 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:04.844 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:04.859 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:04.901 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:04.896+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:04.928 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:04.985 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1] out_osds: [7, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:04.985 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:09:04.986 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T09:09:04.986 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:09:04.986 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:09:04.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T09:09:05.038 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:05.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.812195463999998, 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.039 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:09:05.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.812164613999997, 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.044 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:09:05.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.812138446999995, 2024-09-13T09:09:05.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:09:05.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:09:05.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.812098973000005, 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.053 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:05.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.812064418999995, 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:09:05.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:09:05.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:09:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:09:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.063 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.812032063000004, 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:09:05.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:09:05.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:09:05.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:05.068 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.811999791999995, 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:05.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:05.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.811965756000006, 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.073 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.811932491999997, 2024-09-13T09:09:05.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:09:05.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.811900252000001, 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.083 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:09:05.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.088 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.802920166000007, 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:09:05.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:09:05.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.802879286000007, 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.802839018, 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.099 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:09:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:09:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:09:05.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:05.104 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.802791975000005, 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.109 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.802746412999994, 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:09:05.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:09:05.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.114 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.802656244999994, 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:05.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:09:05.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:05.119 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.802609691000001, 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:05.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.123 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.800968767000001, 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.124 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:09:05.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:09:05.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:05.128 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 93.800902961000006, 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:09:05.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:09:05.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.117895316000002, 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:05.133 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:05.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:05.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:09:05.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:05.138 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:09:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:09:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:05.139 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:05.139 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:05.140 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:05.038+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:05.140 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:05.141 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:09:05.141 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:05.141 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:05.138+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:05.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:05.241 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:05.236+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:05.281 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:05.281 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:05.282 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:05.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:05.342 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:05.336+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:05.353 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-13T09:09:05.425 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:05.425 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:05.425 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:05.442 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:05.443 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:05.436+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:05.544 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:05.542+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:05.566 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451968561000001, 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:09:05.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451947794000006, 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:09:05.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:05.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451929942000007, 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:05.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451913375999993, 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:05.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:09:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:09:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:09:05.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:05.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451887791000004, 2024-09-13T09:09:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:09:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.587 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:05.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:09:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:09:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:09:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:09:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451868623999999, 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:09:05.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:09:05.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451851340999994, 2024-09-13T09:09:05.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:05.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:09:05.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451322470999997, 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:05.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:09:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:09:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:09:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:09:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451290565999997, 2024-09-13T09:09:05.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:05.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:09:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:09:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:09:05.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.451265617999994, 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:09:05.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:09:05.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.619 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.450961053, 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.620 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:09:05.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:09:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:09:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.449198629999998, 2024-09-13T09:09:05.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.449177779999999, 2024-09-13T09:09:05.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:09:05.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.449155456, 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:09:05.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:09:05.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:09:05.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.449136766999999, 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:09:05.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:09:05.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.449119714999995, 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:09:05.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.449102690999993, 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:09:05.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.439089703999997, 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:09:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.058935180999999, 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:09:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:09:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.058754758999999, 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:09:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:05.671 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:05.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:5016 read 0xe1581f thru 0xe1d62e (0x7e10 bytes) 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:5017 punch from 0xde83100 to 0xde84454, (0x1354 bytes) 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:5020 read 0x767bec9 thru 0x7683978 (0x7ab0 bytes) 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:5021 punch from 0x5bbb05b to 0x5bc25c4, (0x7569 bytes) 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:5024 read 0x3d2c267 thru 0x3d38472 (0xc20c bytes) 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:5026 writesame 0x7bb550f thru 0x7bb602a data_size 0x12(0xb1c bytes) 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:5028 write 0x3bb25d thru 0x3c748e (0xc232 bytes) 2024-09-13T09:09:05.674 INFO:teuthology.orchestra.run.smithi031.stdout:5030 write 0x5400627 thru 0x540a8f7 (0xa2d1 bytes) 2024-09-13T09:09:05.675 INFO:teuthology.orchestra.run.smithi031.stdout:5031 writesame 0x144d7a9 thru 0x1455724 data_size 0x52(0x7f7c bytes) 2024-09-13T09:09:05.675 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:05.670+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:05.676 INFO:teuthology.orchestra.run.smithi031.stdout:5032 punch from 0x3713efe to 0x37216fa, (0xd7fc bytes) 2024-09-13T09:09:05.676 INFO:teuthology.orchestra.run.smithi031.stdout:5035 trunc from 0xed2698c to 0xe933d14 2024-09-13T09:09:05.713 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:05.714 INFO:teuthology.orchestra.run.smithi031.stdout:5037 punch from 0x8882342 to 0x888f61d, (0xd2db bytes) 2024-09-13T09:09:05.714 INFO:teuthology.orchestra.run.smithi031.stdout:5041 read 0x527d172 thru 0x528960b (0xc49a bytes) 2024-09-13T09:09:05.736 INFO:teuthology.orchestra.run.smithi031.stdout:5043 trunc from 0xe933d14 to 0xe99078 2024-09-13T09:09:05.773 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:05.770+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:05.813 INFO:teuthology.orchestra.run.smithi031.stdout:5044 trunc from 0xe99078 to 0x694cf26 2024-09-13T09:09:05.814 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:05.814 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:05.814 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:05.814 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:05.828 INFO:teuthology.orchestra.run.smithi031.stdout:5046 read 0x17a04e3 thru 0x17b019e (0xfcbc bytes) 2024-09-13T09:09:05.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:05.841 INFO:teuthology.orchestra.run.smithi031.stdout:5047 trunc from 0x694cf26 to 0x31dafb 2024-09-13T09:09:05.887 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:05.886+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:05.927 INFO:teuthology.orchestra.run.smithi031.stdout:5049 write 0x81e25df thru 0x81e6194 (0x3bb6 bytes) 2024-09-13T09:09:05.934 INFO:teuthology.orchestra.run.smithi031.stdout:5050 write 0x480ab13 thru 0x480ffc5 (0x54b3 bytes) 2024-09-13T09:09:05.965 INFO:teuthology.orchestra.run.smithi031.stdout:5051 writesame 0xc8b456b thru 0xc8b4b9c data_size 0xd(0x632 bytes) 2024-09-13T09:09:05.976 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:05.974+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:05.976 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:05.976 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:05.982 INFO:teuthology.orchestra.run.smithi031.stdout:5053 writesame 0xe436d86 thru 0xe4397cd data_size 0xc(0x2a48 bytes) 2024-09-13T09:09:05.982 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:05.982 INFO:teuthology.orchestra.run.smithi031.stdout:5055 read 0x2a38c86 thru 0x2a3afc8 (0x2343 bytes) 2024-09-13T09:09:05.982 INFO:teuthology.orchestra.run.smithi031.stdout:5056 write 0x7177f66 thru 0x7180660 (0x86fb bytes) 2024-09-13T09:09:05.990 INFO:teuthology.orchestra.run.smithi031.stdout:5057 clone 41 order 21 su 262144 sc 5 2024-09-13T09:09:05.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:06.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:09:06.083 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:06.082+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:06.136 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone40 2024-09-13T09:09:06.175 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.220921919000006, 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:09:06.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:09:06.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88209 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24748032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938372327000003, 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033308870999999997, 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88209 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235077+0000", 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.22e-06 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235080+0000", 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235083+0000", 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268290+0000", 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033207532999999997 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268342+0000", 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999997e-05 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268385+0000", 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2874999999999999e-05 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88210 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24772608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938363856000002, 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033438816000000003, 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88210 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.186 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235084+0000", 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235086+0000", 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235088+0000", 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235091+0000", 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4839999999999998e-06 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268421+0000", 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033329923999999997 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268479+0000", 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8529999999999997e-05 2024-09-13T09:09:06.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268523+0000", 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4048999999999998e-05 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88211 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24797184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938355391000002, 2024-09-13T09:09:06.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033554247000000002, 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88211 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235094+0000", 2024-09-13T09:09:06.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2300000000000001e-06 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235097+0000", 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.021e-06 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235099+0000", 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268550+0000", 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033450510000000003 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268593+0000", 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3035999999999999e-05 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268647+0000", 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4336999999999998e-05 2024-09-13T09:09:06.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88212 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24821760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938347002999997, 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033675415, 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88212 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:06.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235102+0000", 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235105+0000", 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235108+0000", 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.367e-06 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268686+0000", 2024-09-13T09:09:06.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033577810999999999 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268737+0000", 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1134999999999997e-05 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268777+0000", 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0187e-05 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88213 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24846336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938337799999999, 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033785911000000002, 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88213 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235110+0000", 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235112+0000", 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.728e-06 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235119+0000", 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235124+0000", 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268806+0000", 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033682479000000001 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268855+0000", 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999998e-05 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268897+0000", 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1915000000000001e-05 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88214 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24870912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938320165, 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033897243000000001, 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88214 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.205 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235130+0000", 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235139+0000", 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235151+0000", 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1856e-05 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268933+0000", 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033782874999999997 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268986+0000", 2024-09-13T09:09:06.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2593000000000003e-05 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269026+0000", 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9546999999999999e-05 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88215 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24895488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938294777999999, 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033999110999999999, 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88215 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235153+0000", 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235155+0000", 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8810000000000001e-06 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235159+0000", 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235161+0000", 2024-09-13T09:09:06.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269056+0000", 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033895001000000001 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269109+0000", 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269153+0000", 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88216 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24920064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938285020000002, 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120667, 2024-09-13T09:09:06.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88216 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235173+0000", 2024-09-13T09:09:06.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8490000000000001e-06 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235178+0000", 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235180+0000", 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269188+0000", 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034007280000000001 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269245+0000", 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6891000000000001e-05 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269284+0000", 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9561999999999998e-05 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88217 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24944640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938265606000002, 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034230180999999998, 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88217 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235184+0000", 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3120000000000001e-06 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235187+0000", 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235192+0000", 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269313+0000", 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120708 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269363+0000", 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9453999999999998e-05 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269413+0000", 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0531000000000001e-05 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88218 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24969216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938254049000001, 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034354619000000003, 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88218 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235195+0000", 2024-09-13T09:09:06.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-13T09:09:06.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235199+0000", 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235201+0000", 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269450+0000", 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034248860999999999 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269502+0000", 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2519000000000003e-05 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269549+0000", 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6740999999999999e-05 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88219 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24993792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938245508000001, 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034466610000000002, 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88219 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235204+0000", 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.184e-06 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235207+0000", 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-13T09:09:06.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235209+0000", 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269577+0000", 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034368528000000002 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269628+0000", 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0241999999999998e-05 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269670+0000", 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1875999999999997e-05 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.233 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88220 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25018368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938237657999998, 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034591746999999999, 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88220 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235212+0000", 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0839999999999999e-06 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235215+0000", 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9830000000000001e-06 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235217+0000", 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269707+0000", 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034489989999999998 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269762+0000", 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000003e-05 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269803+0000", 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0181e-05 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88221 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25042944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938229575000001, 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034709489000000003, 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88221 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235220+0000", 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0640000000000001e-06 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235223+0000", 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235225+0000", 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269830+0000", 2024-09-13T09:09:06.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034604633000000003 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269884+0000", 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067000000000003e-05 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269928+0000", 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4607000000000002e-05 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88222 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25067520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938221429000002, 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034835487999999998, 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88222 2024-09-13T09:09:06.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:06.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235228+0000", 2024-09-13T09:09:06.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235231+0000", 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235233+0000", 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:09:06.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269963+0000", 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034730039999999997 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270016+0000", 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2581999999999999e-05 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270063+0000", 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6718000000000002e-05 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88223 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25092096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938213208000001, 2024-09-13T09:09:06.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034953250999999998, 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88223 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:06.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235236+0000", 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.127e-06 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235239+0000", 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0180000000000002e-06 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235241+0000", 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270091+0000", 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034849036 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270137+0000", 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6112999999999997e-05 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270189+0000", 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88224 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25116672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938205009000001, 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035071738999999998, 2024-09-13T09:09:06.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88224 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235244+0000", 2024-09-13T09:09:06.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235247+0000", 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7750000000000001e-06 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235249+0000", 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270226+0000", 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034977019999999998 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270275+0000", 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8791000000000001e-05 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270315+0000", 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9953999999999997e-05 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88225 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25141248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.938197097, 2024-09-13T09:09:06.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035182109000000003, 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88225 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:06.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:06.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235252+0000", 2024-09-13T09:09:06.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:06.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235255+0000", 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235258+0000", 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270337+0000", 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035079359999999997 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270388+0000", 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0986000000000003e-05 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270433+0000", 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5581000000000003e-05 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93981 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [watch reconnect cookie 140193706156944 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.319349559000003, 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116345170000001, 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93981 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854096+0000", 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854113+0000", 2024-09-13T09:09:06.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6759999999999999e-05 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854146+0000", 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3506000000000001e-05 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854169+0000", 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2331999999999999e-05 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856384+0000", 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022150960000000002 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856397+0000", 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3047999999999999e-05 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863781+0000", 2024-09-13T09:09:06.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007384276 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863918+0000", 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013628800000000001 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865733+0000", 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018159109999999999 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93982 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.315685627000001, 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0094988519999999, 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93982 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857761+0000", 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857770+0000", 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2109999999999998e-06 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857789+0000", 2024-09-13T09:09:06.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9111999999999999e-05 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857800+0000", 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857854+0000", 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4030999999999998e-05 2024-09-13T09:09:06.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857866+0000", 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2275e-05 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864650+0000", 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0067835570000001 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864768+0000", 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118019 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865240+0000", 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047203799999999998 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.866250+0000", 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001009935 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867165+0000", 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091516600000000003 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867219+0000", 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4330999999999999e-05 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867262+0000", 2024-09-13T09:09:06.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.265e-05 2024-09-13T09:09:06.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.277 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:06.279 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:06.279 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:06.272+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:06.354 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-13T09:09:06.379 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:06.372+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:06.452 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:06.452 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:06.452 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:06.452 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:06.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:06.480 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:06.472+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:06.581 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:06.578+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:06.653 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:06.653 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:06.653 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:06.653 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:06.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:06.715 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:06.714+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:06.783 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:06.780+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:06.840 INFO:teuthology.orchestra.run.smithi031.stdout:5058 punch from 0xe1b1a1 to 0xe2572b, (0xa58a bytes) 2024-09-13T09:09:06.849 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:06.849 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:06.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:06.849 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:06.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.849 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.849 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503487871999994, 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503471098999995, 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:09:06.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:09:06.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.859 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503433794000003, 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:09:06.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:09:06.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503414079999999, 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:09:06.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:09:06.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503397754999995, 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:06.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:09:06.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503381888999996, 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:09:06.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:09:06.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503366103999994, 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:09:06.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:09:06.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503347032999997, 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:09:06.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:09:06.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503330485999996, 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.894 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503302887000004, 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:09:06.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503285645000005, 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:09:06.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:09:06.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503255202000005, 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:09:06.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:09:06.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503236286999993, 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:09:06.910 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:09:06.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503218498999999, 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:09:06.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:06.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503193275000001, 2024-09-13T09:09:06.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:09:06.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503175071000001, 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:09:06.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503154753000004, 2024-09-13T09:09:06.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:06.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503136205000004, 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:09:06.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:09:06.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503118931000003, 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:06.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:09:06.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.945 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.503101477000001, 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:06.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:06.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:09:06.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.950 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:06.951 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:06.951 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:06.951 INFO:teuthology.orchestra.run.smithi031.stdout:5059 punch from 0x87632e0 to 0x8770426, (0xd146 bytes) 2024-09-13T09:09:06.951 INFO:teuthology.orchestra.run.smithi031.stdout:5060 write 0xd91fb64 thru 0xd921be4 (0x2081 bytes) 2024-09-13T09:09:06.951 INFO:teuthology.orchestra.run.smithi031.stdout:5063 write 0x7189b59 thru 0x719858c (0xea34 bytes) 2024-09-13T09:09:06.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:06.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:5064 read 0x6fb486c thru 0x6fc39e6 (0xf17b bytes) 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:5065 read 0x7b68d77 thru 0x7b6cc8d (0x3f17 bytes) 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:5066 write 0x54ad9c8 thru 0x54ae686 (0xcbf bytes) 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:5070 trunc from 0xe4397ce to 0xa6f1fb8 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:5072 punch from 0x8eb3068 to 0x8eb51ec, (0x2184 bytes) 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:5073 punch from 0x6a4ee7e to 0x6a59ab2, (0xac34 bytes) 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:5076 punch from 0x28dbbb6 to 0x28dd273, (0x16bd bytes) 2024-09-13T09:09:06.956 INFO:teuthology.orchestra.run.smithi031.stdout:5077 write 0x7cb0e40 thru 0x7cb4838 (0x39f9 bytes) 2024-09-13T09:09:06.957 INFO:teuthology.orchestra.run.smithi031.stdout:5078 writesame 0xe03cd76 thru 0xe048d54 data_size 0x7(0xbfdf bytes) 2024-09-13T09:09:06.957 INFO:teuthology.orchestra.run.smithi031.stdout:5079 read 0x96bea33 thru 0x96cb22e (0xc7fc bytes) 2024-09-13T09:09:06.957 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:06.957 INFO:teuthology.orchestra.run.smithi031.stdout:5081 read 0xd49ab4 thru 0xd53b72 (0xa0bf bytes) 2024-09-13T09:09:06.957 INFO:teuthology.orchestra.run.smithi031.stdout:5082 writesame 0x77b5a99 thru 0x77b662d data_size 0x1(0xb95 bytes) 2024-09-13T09:09:06.957 INFO:teuthology.orchestra.run.smithi031.stdout:5083 punch from 0x485b82c to 0x4867749, (0xbf1d bytes) 2024-09-13T09:09:06.957 INFO:teuthology.orchestra.run.smithi031.stdout:5084 write 0x5a16bd4 thru 0x5a1dec4 (0x72f1 bytes) 2024-09-13T09:09:06.958 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:06.948+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:06.960 INFO:teuthology.orchestra.run.smithi031.stdout:5085 write 0x7e631cf thru 0x7e6cc7c (0x9aae bytes) 2024-09-13T09:09:06.968 INFO:teuthology.orchestra.run.smithi031.stdout:5086 write 0x84feea2 thru 0x850bce3 (0xce42 bytes) 2024-09-13T09:09:06.977 INFO:teuthology.orchestra.run.smithi031.stdout:5088 write 0x5344653 thru 0x534f00d (0xa9bb bytes) 2024-09-13T09:09:06.996 INFO:teuthology.orchestra.run.smithi031.stdout:5090 trunc from 0xe048d55 to 0x616d286 2024-09-13T09:09:07.027 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:09:07.042 INFO:teuthology.orchestra.run.smithi031.stdout:5091 read 0x263ce5e thru 0x26438c2 (0x6a65 bytes) 2024-09-13T09:09:07.043 INFO:teuthology.orchestra.run.smithi031.stdout:5092 punch from 0x4f98d1b to 0x4f9bb3b, (0x2e20 bytes) 2024-09-13T09:09:07.043 INFO:teuthology.orchestra.run.smithi031.stdout:5093 clone 42 order 19 su 8192 sc 7 2024-09-13T09:09:07.056 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:07.054+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:07.111 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:07.111 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:07.111 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:07.111 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:07.112 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:07.113 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:07.126 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:07.130 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:07.130 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:07.147 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:07.147 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:07.158 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:07.156+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:07.164 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:07.164 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:07.183 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:07.184 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:07.204 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:07.204 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:07.224 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:07.243 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:07.243 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:07.244 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:07.244 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:07.259 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:07.252+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:07.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:07.359 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:07.352+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:07.433 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:07.433 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:07.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:07.461 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:07.458+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:07.562 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:07.558+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:07.582 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:07.582 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:07.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:07.663 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:07.662+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:07.716 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:07.716 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:07.730 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:07.777 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:07.772+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:07.849 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:07.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:07.865 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:07.862+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:07.966 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:07.960+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:08.040 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:08.040 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:08.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:08.067 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:08.066+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:08.154 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone41 intact 2024-09-13T09:09:08.168 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:08.166+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:08.215 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:08.215 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:08.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:08.269 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:08.266+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:08.295 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-13T09:09:08.359 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:08.360 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:08.370 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:08.366+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:08.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:08.471 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:08.470+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:08.510 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:08.510 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:08.524 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:08.572 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:08.570+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:08.651 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:08.651 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:08.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:08.717 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:08.712+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:08.775 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:08.770+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:08.791 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:08.792 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:08.805 INFO:teuthology.orchestra.run.smithi031.stdout:5094 read 0x14585c thru 0x154ea0 (0xf645 bytes) 2024-09-13T09:09:08.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:08.809 INFO:teuthology.orchestra.run.smithi031.stdout:5095 punch from 0x4a62878 to 0x4a6e6c4, (0xbe4c bytes) 2024-09-13T09:09:08.815 INFO:teuthology.orchestra.run.smithi031.stdout:5096 punch from 0x5ac3bd5 to 0x5acc7b1, (0x8bdc bytes) 2024-09-13T09:09:08.815 INFO:teuthology.orchestra.run.smithi031.stdout:5097 read 0x485a231 thru 0x486088e (0x665e bytes) 2024-09-13T09:09:08.815 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:08.815 INFO:teuthology.orchestra.run.smithi031.stdout:5099 punch from 0x48b8012 to 0x48bb5df, (0x35cd bytes) 2024-09-13T09:09:08.815 INFO:teuthology.orchestra.run.smithi031.stdout:5100 punch from 0x5e36684 to 0x5e3b70b, (0x5087 bytes) 2024-09-13T09:09:08.832 INFO:teuthology.orchestra.run.smithi031.stdout:5101 writesame 0xb136c91 thru 0xb13b03e data_size 0x1(0x43ae bytes) 2024-09-13T09:09:08.833 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:08.833 INFO:teuthology.orchestra.run.smithi031.stdout:5103 punch from 0x69e8b83 to 0x69ec710, (0x3b8d bytes) 2024-09-13T09:09:08.833 INFO:teuthology.orchestra.run.smithi031.stdout:5105 write 0x7087be0 thru 0x70961fa (0xe61b bytes) 2024-09-13T09:09:08.847 INFO:teuthology.orchestra.run.smithi031.stdout:5107 write 0x1eb25a2 thru 0x1eb8ac8 (0x6527 bytes) 2024-09-13T09:09:08.860 INFO:teuthology.orchestra.run.smithi031.stdout:5110 punch from 0xa6dbc65 to 0xa6e2b61, (0x6efc bytes) 2024-09-13T09:09:08.862 INFO:teuthology.orchestra.run.smithi031.stdout:5111 trunc from 0xb13b03f to 0xc19cc00 2024-09-13T09:09:08.872 INFO:teuthology.orchestra.run.smithi031.stdout:5113 punch from 0xbe2bfbd to 0xbe3581c, (0x985f bytes) 2024-09-13T09:09:08.875 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:08.874+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:5115 write 0xdd90845 thru 0xdd9bfa4 (0xb760 bytes) 2024-09-13T09:09:08.907 INFO:teuthology.orchestra.run.smithi031.stdout:5117 trunc from 0xdd9bfa5 to 0xa937504 2024-09-13T09:09:08.948 INFO:teuthology.orchestra.run.smithi031.stdout:5118 write 0xcb33353 thru 0xcb3a75b (0x7409 bytes) 2024-09-13T09:09:08.961 INFO:teuthology.orchestra.run.smithi031.stdout:5119 write 0x88b5c29 thru 0x88bec0b (0x8fe3 bytes) 2024-09-13T09:09:08.975 INFO:teuthology.orchestra.run.smithi031.stdout:5120 punch from 0x8e80723 to 0x8e8bbf3, (0xb4d0 bytes) 2024-09-13T09:09:08.976 INFO:teuthology.orchestra.run.smithi031.stdout:5121 read 0xca91a20 thru 0xca9b223 (0x9804 bytes) 2024-09-13T09:09:08.976 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:08.976 INFO:teuthology.orchestra.run.smithi031.stdout:5124 read 0x20c7e41 thru 0x20cddf1 (0x5fb1 bytes) 2024-09-13T09:09:08.976 INFO:teuthology.orchestra.run.smithi031.stdout:5125 punch from 0x139c21e to 0x139d25d, (0x103f bytes) 2024-09-13T09:09:08.976 INFO:teuthology.orchestra.run.smithi031.stdout:5128 writesame 0xace7dd0 thru 0xace8e05 data_size 0x19(0x1036 bytes) 2024-09-13T09:09:08.976 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:08.974+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:08.984 INFO:teuthology.orchestra.run.smithi031.stdout:5129 punch from 0x4cd82b6 to 0x4cdddf2, (0x5b3c bytes) 2024-09-13T09:09:08.984 INFO:teuthology.orchestra.run.smithi031.stdout:5130 write 0x816b898 thru 0x8172a86 (0x71ef bytes) 2024-09-13T09:09:08.996 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:08.996 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:08.997 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:08.997 INFO:teuthology.orchestra.run.smithi031.stdout:5134 read 0xc5f82b1 thru 0xc6074aa (0xf1fa bytes) 2024-09-13T09:09:08.997 INFO:teuthology.orchestra.run.smithi031.stdout:5136 write 0xa72f9f6 thru 0xa7333e8 (0x39f3 bytes) 2024-09-13T09:09:09.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:09.023 INFO:teuthology.orchestra.run.smithi031.stdout:5137 writesame 0xdb8bed3 thru 0xdb8eb6c data_size 0x21(0x2c9a bytes) 2024-09-13T09:09:09.023 INFO:teuthology.orchestra.run.smithi031.stdout:5138 read 0x6768668 thru 0x67722ee (0x9c87 bytes) 2024-09-13T09:09:09.024 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.024 INFO:teuthology.orchestra.run.smithi031.stdout:5141 punch from 0xba161d3 to 0xba1be22, (0x5c4f bytes) 2024-09-13T09:09:09.024 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.024 INFO:teuthology.orchestra.run.smithi031.stdout:5143 punch from 0xcb1e0e3 to 0xcb28c08, (0xab25 bytes) 2024-09-13T09:09:09.024 INFO:teuthology.orchestra.run.smithi031.stdout:5145 writesame 0x5448c9 thru 0x545b40 data_size 0x3(0x1278 bytes) 2024-09-13T09:09:09.039 INFO:teuthology.orchestra.run.smithi031.stdout:5146 write 0x4ba9a97 thru 0x4bb2ae3 (0x904d bytes) 2024-09-13T09:09:09.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:09:09.156 INFO:teuthology.orchestra.run.smithi031.stdout:5148 trunc from 0xdb8eb6d to 0x9973fc3 2024-09-13T09:09:09.156 INFO:teuthology.orchestra.run.smithi031.stdout:5150 read 0x2dba78d thru 0x2dbfd7d (0x55f1 bytes) 2024-09-13T09:09:09.156 INFO:teuthology.orchestra.run.smithi031.stdout:5151 writesame 0x802cd7d thru 0x803a388 data_size 0x67(0xd60c bytes) 2024-09-13T09:09:09.156 INFO:teuthology.orchestra.run.smithi031.stdout:5153 read 0xff2e6f thru 0xfff448 (0xc5da bytes) 2024-09-13T09:09:09.156 INFO:teuthology.orchestra.run.smithi031.stdout:5155 read 0x2623817 thru 0x2632382 (0xeb6c bytes) 2024-09-13T09:09:09.156 INFO:teuthology.orchestra.run.smithi031.stdout:5157 write 0x890805a thru 0x890951c (0x14c3 bytes) 2024-09-13T09:09:09.156 INFO:teuthology.orchestra.run.smithi031.stdout:5158 punch from 0x5647a8c to 0x5655ea1, (0xe415 bytes) 2024-09-13T09:09:09.156 INFO:teuthology.orchestra.run.smithi031.stdout:5159 write 0x3513832 thru 0x352195a (0xe129 bytes) 2024-09-13T09:09:09.157 INFO:teuthology.orchestra.run.smithi031.stdout:5160 punch from 0x78f274f to 0x78fd27b, (0xab2c bytes) 2024-09-13T09:09:09.157 INFO:teuthology.orchestra.run.smithi031.stdout:5161 read 0x6597277 thru 0x65a2fc9 (0xbd53 bytes) 2024-09-13T09:09:09.157 INFO:teuthology.orchestra.run.smithi031.stdout:5162 punch from 0x2bba9b2 to 0x2bc101e, (0x666c bytes) 2024-09-13T09:09:09.157 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.157 INFO:teuthology.orchestra.run.smithi031.stdout:5166 read 0x724a8b1 thru 0x7258308 (0xda58 bytes) 2024-09-13T09:09:09.157 INFO:teuthology.orchestra.run.smithi031.stdout:5167 writesame 0xa95d304 thru 0xa9686ff data_size 0x2(0xb3fc bytes) 2024-09-13T09:09:09.157 INFO:teuthology.orchestra.run.smithi031.stdout:5169 writesame 0x1fadc67 thru 0x1fba993 data_size 0x37(0xcd2d bytes) 2024-09-13T09:09:09.158 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:09.148+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:09.162 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:09.162 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:09.164 INFO:teuthology.orchestra.run.smithi031.stdout:5170 read 0x1790b72 thru 0x179a4db (0x996a bytes) 2024-09-13T09:09:09.164 INFO:teuthology.orchestra.run.smithi031.stdout:5171 read 0x74406ca thru 0x744fdde (0xf715 bytes) 2024-09-13T09:09:09.165 INFO:teuthology.orchestra.run.smithi031.stdout:5173 read 0xa81f267 thru 0xa82544d (0x61e7 bytes) 2024-09-13T09:09:09.165 INFO:teuthology.orchestra.run.smithi031.stdout:5174 punch from 0x249754a to 0x249806a, (0xb20 bytes) 2024-09-13T09:09:09.166 INFO:teuthology.orchestra.run.smithi031.stdout:5175 punch from 0x836a96a to 0x8379d94, (0xf42a bytes) 2024-09-13T09:09:09.166 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.166 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.166 INFO:teuthology.orchestra.run.smithi031.stdout:5178 writesame 0xa8c60a3 thru 0xa8cd283 data_size 0x1(0x71e1 bytes) 2024-09-13T09:09:09.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:09.206 INFO:teuthology.orchestra.run.smithi031.stdout:5179 write 0x814dcfb thru 0x8159f77 (0xc27d bytes) 2024-09-13T09:09:09.210 INFO:teuthology.orchestra.run.smithi031.stdout:5180 punch from 0xa2ea52b to 0xa2f6da6, (0xc87b bytes) 2024-09-13T09:09:09.214 INFO:teuthology.orchestra.run.smithi031.stdout:5181 trunc from 0xa968700 to 0x885d309 2024-09-13T09:09:09.232 INFO:teuthology.orchestra.run.smithi031.stdout:5184 write 0x774ae64 thru 0x775957f (0xe71c bytes) 2024-09-13T09:09:09.246 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.256 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:09.254+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:09.256 INFO:teuthology.orchestra.run.smithi031.stdout:5189 write 0x959ef4e thru 0x95abb7b (0xcc2e bytes) 2024-09-13T09:09:09.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:09.284 INFO:teuthology.orchestra.run.smithi031.stdout:5192 trunc from 0x95abb7c to 0xdc333b 2024-09-13T09:09:09.323 INFO:teuthology.orchestra.run.smithi031.stdout:5193 read 0x2e2bbb thru 0x2e41e2 (0x1628 bytes) 2024-09-13T09:09:09.324 INFO:teuthology.orchestra.run.smithi031.stdout:5194 read 0x41e9ae thru 0x4238ae (0x4f01 bytes) 2024-09-13T09:09:09.324 INFO:teuthology.orchestra.run.smithi031.stdout:5197 read 0x17fe81 thru 0x18d334 (0xd4b4 bytes) 2024-09-13T09:09:09.337 INFO:teuthology.orchestra.run.smithi031.stdout:5198 write 0x4006aa0 thru 0x4007cd3 (0x1234 bytes) 2024-09-13T09:09:09.355 INFO:teuthology.orchestra.run.smithi031.stdout:5199 trunc from 0x4007cd4 to 0xe2c7ca2 2024-09-13T09:09:09.356 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:09.348+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:09.361 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:09.361 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:09.363 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.363 INFO:teuthology.orchestra.run.smithi031.stdout:5201 write 0x71a096b thru 0x71af962 (0xeff8 bytes) 2024-09-13T09:09:09.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:09.378 INFO:teuthology.orchestra.run.smithi031.stdout:5204 trunc from 0xe2c7ca2 to 0xaefc6b1 2024-09-13T09:09:09.387 INFO:teuthology.orchestra.run.smithi031.stdout:5205 read 0x28a66fb thru 0x28b00f9 (0x99ff bytes) 2024-09-13T09:09:09.389 INFO:teuthology.orchestra.run.smithi031.stdout:5207 trunc from 0xaefc6b1 to 0xa0dd000 2024-09-13T09:09:09.399 INFO:teuthology.orchestra.run.smithi031.stdout:5208 write 0x22352cb thru 0x2236f6f (0x1ca5 bytes) 2024-09-13T09:09:09.416 INFO:teuthology.orchestra.run.smithi031.stdout:5209 trunc from 0xa0dd000 to 0x37ed58f 2024-09-13T09:09:09.435 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.444 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:09.444 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:09.450 INFO:teuthology.orchestra.run.smithi031.stdout:5211 write 0x56b931f thru 0x56bcff4 (0x3cd6 bytes) 2024-09-13T09:09:09.457 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:09.452+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:09.460 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:09.460 INFO:teuthology.orchestra.run.smithi031.stdout:5213 clone 43 order 20 su 32768 sc 5 2024-09-13T09:09:09.463 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:09.463 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:09.480 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:09.481 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:09.497 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:09.506 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:09.506 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:09.506 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:09.506 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:09.517 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:09.517 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:09.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:09.538 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:09.538 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:09.558 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:09.558 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:09.558 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:09.554+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:09.578 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:09.578 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:09.655 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:09.655 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:09.655 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:09.658 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:09.652+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:09.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:09.759 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:09.754+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:09.803 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:09.803 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:09.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:09.803 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:09.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.803 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.687431395000004, 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:09.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:09:09.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.687410627999995, 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.809 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:09:09.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:09:09.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.687392775999996, 2024-09-13T09:09:09.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:09:09.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.687376209999996, 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:09:09.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:09:09.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.687350624999993, 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:09:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:09:09.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.687331458000003, 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:09:09.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:09:09.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.687314174999997, 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:09:09.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.686785305000001, 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:09.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.686753400000001, 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:09:09.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:09:09.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.686728451999997, 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.686423887000004, 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:09:09.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.684661464000001, 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.855 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:09:09.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:09:09.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.684640614000003, 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.860 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:09:09.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.864 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.684618290000003, 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:09:09.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.684599601000002, 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:09:09.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:09:09.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.684582548999998, 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:09:09.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.684565524999996, 2024-09-13T09:09:09.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:09.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.674552538, 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:09:09.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:09:09.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:09.889 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.294398014999999, 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:09.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:09:09.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.894 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.294217592999999, 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:09.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:09:09.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:09:09.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.899 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:09.900 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:09.900 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:09.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:09.902 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:09.898+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:10.002 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:09.996+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:10.043 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:09:10.089 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:10.089 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:10.089 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:10.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:10.104 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:10.096+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:10.148 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1] out_osds: [7, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:10.148 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:09:10.149 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:09:10.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:09:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone42 from 0x56bcff5 (overlap 0xdc333b) to 0xc62e1b 2024-09-13T09:09:10.205 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:10.200+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:10.276 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:10.277 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:10.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:10.305 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:10.300+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-13T09:09:10.406 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:10.402+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:10.455 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:10.455 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.499804139000005, 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:10.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:09:10.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88209 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24748032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217254546999996, 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033308870999999997, 2024-09-13T09:09:10.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88209 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235077+0000", 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.22e-06 2024-09-13T09:09:10.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235080+0000", 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235083+0000", 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268290+0000", 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033207532999999997 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268342+0000", 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999997e-05 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268385+0000", 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2874999999999999e-05 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88210 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24772608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217246075999995, 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033438816000000003, 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88210 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235084+0000", 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235086+0000", 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235088+0000", 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:09:10.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235091+0000", 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4839999999999998e-06 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268421+0000", 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033329923999999997 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268479+0000", 2024-09-13T09:09:10.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8529999999999997e-05 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268523+0000", 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4048999999999998e-05 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88211 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24797184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:10.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217237611000002, 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033554247000000002, 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88211 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:10.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235094+0000", 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2300000000000001e-06 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235097+0000", 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.021e-06 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235099+0000", 2024-09-13T09:09:10.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268550+0000", 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033450510000000003 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268593+0000", 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3035999999999999e-05 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268647+0000", 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4336999999999998e-05 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88212 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24821760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217229223000004, 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033675415, 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88212 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235102+0000", 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-13T09:09:10.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235105+0000", 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235108+0000", 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.367e-06 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268686+0000", 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033577810999999999 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268737+0000", 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1134999999999997e-05 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268777+0000", 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0187e-05 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88213 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24846336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217220019999999, 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033785911000000002, 2024-09-13T09:09:10.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88213 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235110+0000", 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235112+0000", 2024-09-13T09:09:10.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.728e-06 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235119+0000", 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235124+0000", 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268806+0000", 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033682479000000001 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268855+0000", 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999998e-05 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268897+0000", 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1915000000000001e-05 2024-09-13T09:09:10.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88214 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24870912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217202384999993, 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033897243000000001, 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.485 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88214 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:10.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235130+0000", 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235139+0000", 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235151+0000", 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1856e-05 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268933+0000", 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033782874999999997 2024-09-13T09:09:10.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268986+0000", 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2593000000000003e-05 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269026+0000", 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9546999999999999e-05 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88215 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24895488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217176997999999, 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033999110999999999, 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88215 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.490 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235153+0000", 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235155+0000", 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8810000000000001e-06 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235159+0000", 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:09:10.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235161+0000", 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269056+0000", 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033895001000000001 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269109+0000", 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269153+0000", 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88216 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24920064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.494 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217167239999995, 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120667, 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88216 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:10.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235173+0000", 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8490000000000001e-06 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235178+0000", 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235180+0000", 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269188+0000", 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034007280000000001 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269245+0000", 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6891000000000001e-05 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269284+0000", 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9561999999999998e-05 2024-09-13T09:09:10.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88217 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24944640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217147826000001, 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034230180999999998, 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.499 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88217 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:10.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235184+0000", 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3120000000000001e-06 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235187+0000", 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235192+0000", 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269313+0000", 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120708 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269363+0000", 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9453999999999998e-05 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269413+0000", 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0531000000000001e-05 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88218 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24969216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217136268999994, 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034354619000000003, 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88218 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235195+0000", 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235199+0000", 2024-09-13T09:09:10.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235201+0000", 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269450+0000", 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034248860999999999 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269502+0000", 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2519000000000003e-05 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269549+0000", 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6740999999999999e-05 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88219 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24993792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217127727999994, 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034466610000000002, 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88219 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:10.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235204+0000", 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.184e-06 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235207+0000", 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235209+0000", 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269577+0000", 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034368528000000002 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269628+0000", 2024-09-13T09:09:10.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0241999999999998e-05 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269670+0000", 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1875999999999997e-05 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88220 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25018368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.513 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217119878000005, 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034591746999999999, 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88220 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235212+0000", 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0839999999999999e-06 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235215+0000", 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9830000000000001e-06 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235217+0000", 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269707+0000", 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034489989999999998 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269762+0000", 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000003e-05 2024-09-13T09:09:10.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269803+0000", 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0181e-05 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88221 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25042944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217111794999994, 2024-09-13T09:09:10.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034709489000000003, 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88221 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:10.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235220+0000", 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0640000000000001e-06 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235223+0000", 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235225+0000", 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:09:10.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269830+0000", 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034604633000000003 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269884+0000", 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067000000000003e-05 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269928+0000", 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4607000000000002e-05 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88222 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25067520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217103648999995, 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034835487999999998, 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88222 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235228+0000", 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:10.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235231+0000", 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235233+0000", 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269963+0000", 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034730039999999997 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270016+0000", 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2581999999999999e-05 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270063+0000", 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6718000000000002e-05 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88223 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25092096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217095427999993, 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034953250999999998, 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88223 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235236+0000", 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.127e-06 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235239+0000", 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0180000000000002e-06 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235241+0000", 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270091+0000", 2024-09-13T09:09:10.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034849036 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270137+0000", 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6112999999999997e-05 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270189+0000", 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88224 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25116672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217087229000001, 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035071738999999998, 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88224 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235244+0000", 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235247+0000", 2024-09-13T09:09:10.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7750000000000001e-06 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235249+0000", 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270226+0000", 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034977019999999998 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270275+0000", 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8791000000000001e-05 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270315+0000", 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9953999999999997e-05 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88225 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25141248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.217079317, 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035182109000000003, 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88225 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:10.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235252+0000", 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235255+0000", 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235258+0000", 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270337+0000", 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035079359999999997 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270388+0000", 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0986000000000003e-05 2024-09-13T09:09:10.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270433+0000", 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5581000000000003e-05 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93981 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [watch reconnect cookie 140193706156944 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.598231779000002, 2024-09-13T09:09:10.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116345170000001, 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93981 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854096+0000", 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854113+0000", 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6759999999999999e-05 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854146+0000", 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3506000000000001e-05 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854169+0000", 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2331999999999999e-05 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856384+0000", 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022150960000000002 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856397+0000", 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3047999999999999e-05 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863781+0000", 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007384276 2024-09-13T09:09:10.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863918+0000", 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013628800000000001 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865733+0000", 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018159109999999999 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93982 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.594567847, 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0094988519999999, 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93982 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857761+0000", 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857770+0000", 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2109999999999998e-06 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857789+0000", 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9111999999999999e-05 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857800+0000", 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857854+0000", 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4030999999999998e-05 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857866+0000", 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2275e-05 2024-09-13T09:09:10.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864650+0000", 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0067835570000001 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864768+0000", 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118019 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:09:10.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865240+0000", 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047203799999999998 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.866250+0000", 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001009935 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867165+0000", 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091516600000000003 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867219+0000", 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4330999999999999e-05 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867262+0000", 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.265e-05 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.554 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:10.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:09:10.557 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 114, 2024-09-13T09:09:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:09:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T09:06:44.149006+0000", 2024-09-13T09:09:10.557 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T09:09:10.033795+0000", 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T09:06:55.611487+0000", 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T09:08:59.952135+0000", 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:09:10.558 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:06:55.131330+0000", 2024-09-13T09:09:10.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T09:09:10.560 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.561 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:10.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:10.563 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:10.564 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T09:09:10.565 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:02.743102+0000", 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:09:10.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T09:09:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:10.568 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:10.569 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:10.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:10.571 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9699997901916504, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9699997901916504, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.62999999523162842, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 3.130000114440918, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 3.130000114440918, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:10.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:12.517158+0000", 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:10.573 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:10.574 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "114", 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 46, 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 114, 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:10.575 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:10.576 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:10.577 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.75, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.309999942779541, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.309999942779541, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:10.578 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "33c6fc7d-9828-4500-b103-b96d1cd7f420", 2024-09-13T09:09:10.579 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 107, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:10.582 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.583 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.584 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6818/3291756747", 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6822/3291756747", 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6830/3291756747", 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6826/3291756747", 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "869ea3c5-8e6e-4b49-b70e-083f242a7aae", 2024-09-13T09:09:10.585 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.999481201171875, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 108, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:10.587 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.588 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:10.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/533520566", 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/533520566", 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/533520566", 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/533520566", 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e144d496-76ee-46e0-bdc6-135c77128535", 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:10.592 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:10.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T09:09:10.594 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6819/2253604482", 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6823/2253604482", 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/2253604482", 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6827/2253604482", 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:10.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "696b9414-9c19-4984-a8ff-14548e46b357", 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 108, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:10.598 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.599 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/3367214693", 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/3367214693", 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/3367214693", 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/3367214693", 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ac58040d-9b54-42ca-b143-1816abb54816", 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:10.603 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6800", 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:10.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6801", 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6802", 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:10.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6803", 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6806", 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:10.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6807", 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6804", 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6805", 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6801/3778074750", 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6803/3778074750", 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6807/3778074750", 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6805/3778074750", 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2", 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:10.609 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6808", 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6809", 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.610 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6811", 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6813", 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6819", 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6824", 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6815", 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6817", 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:10.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6809/1926603087", 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6813/1926603087", 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6824/1926603087", 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6817/1926603087", 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:09:10.614 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d", 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6820", 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6821", 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:10.616 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6823", 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6825", 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:10.617 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6830", 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6831", 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:10.618 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6827", 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6828", 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6821/766305500", 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6825/766305500", 2024-09-13T09:09:10.619 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6831/766305500", 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6828/766305500", 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f36c332c-9497-47e8-b943-f6b6dcaebeb8", 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-13T09:09:10.620 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6810", 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:10.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6812", 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6814", 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6816", 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6826", 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6829", 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6818", 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:10.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6822", 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6812/1032956039", 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6816/1032956039", 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6829/1032956039", 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6822/1032956039", 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:10.625 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 16238, 2024-09-13T09:09:10.626 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.183436+0000", 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.264080+0000", 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:10.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.253587+0000", 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:09:10.628 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.244429+0000", 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:10.629 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.369458+0000", 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.158089+0000", 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.217295+0000", 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:10.631 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.332254+0000", 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.a", 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: 4, 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: 2 2024-09-13T09:09:10.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T09:09:10.633 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T09:09:10.634 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T09:09:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:09:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:09:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T09:09:10.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.635 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:10.636 INFO:tasks.thrashosds.thrasher:j is {'epoch': 114, 'fsid': 'fe16bf6d-4166-4417-80fd-1123617a33d3', 'created': '2024-09-13T09:06:44.149006+0000', 'modified': '2024-09-13T09:09:10.033795+0000', 'last_up_change': '2024-09-13T09:06:55.611487+0000', 'last_in_change': '2024-09-13T09:08:59.952135+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:06:55.131330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:07:02.743102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.96999979019165, 'score_stable': 4.96999979019165, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 3.130000114440918, 'raw_score_stable': 3.130000114440918, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:07:12.517158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 0.75, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '33c6fc7d-9828-4500-b103-b96d1cd7f420', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6818', 'nonce': 3291756747}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6822', 'nonce': 3291756747}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6830', 'nonce': 3291756747}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6826', 'nonce': 3291756747}]}, 'public_addr': '172.21.15.31:6818/3291756747', 'cluster_addr': '172.21.15.31:6822/3291756747', 'heartbeat_back_addr': '172.21.15.31:6830/3291756747', 'heartbeat_front_addr': '172.21.15.31:6826/3291756747', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.999481201171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 533520566}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 533520566}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 533520566}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 533520566}]}, 'public_addr': '172.21.15.31:6801/533520566', 'cluster_addr': '172.21.15.31:6803/533520566', 'heartbeat_back_addr': '172.21.15.31:6807/533520566', 'heartbeat_front_addr': '172.21.15.31:6805/533520566', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e144d496-76ee-46e0-bdc6-135c77128535', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6817', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2253604482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6821', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2253604482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6829', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 2253604482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6825', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 2253604482}]}, 'public_addr': '172.21.15.31:6819/2253604482', 'cluster_addr': '172.21.15.31:6823/2253604482', 'heartbeat_back_addr': '172.21.15.31:6831/2253604482', 'heartbeat_front_addr': '172.21.15.31:6827/2253604482', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '696b9414-9c19-4984-a8ff-14548e46b357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 3367214693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 3367214693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 3367214693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 3367214693}]}, 'public_addr': '172.21.15.31:6809/3367214693', 'cluster_addr': '172.21.15.31:6811/3367214693', 'heartbeat_back_addr': '172.21.15.31:6815/3367214693', 'heartbeat_front_addr': '172.21.15.31:6813/3367214693', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ac58040d-9b54-42ca-b143-1816abb54816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6800', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6801', 'nonce': 3778074750}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 3778074750}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 3778074750}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 3778074750}]}, 'public_addr': '172.21.15.60:6801/3778074750', 'cluster_addr': '172.21.15.60:6803/3778074750', 'heartbeat_back_addr': '172.21.15.60:6807/3778074750', 'heartbeat_front_addr': '172.21.15.60:6805/3778074750', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1926603087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6811', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 1926603087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6819', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6824', 'nonce': 1926603087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6815', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 1926603087}]}, 'public_addr': '172.21.15.60:6809/1926603087', 'cluster_addr': '172.21.15.60:6813/1926603087', 'heartbeat_back_addr': '172.21.15.60:6824/1926603087', 'heartbeat_front_addr': '172.21.15.60:6817/1926603087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 766305500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6823', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 766305500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 766305500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6827', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6828', 'nonce': 766305500}]}, 'public_addr': '172.21.15.60:6821/766305500', 'cluster_addr': '172.21.15.60:6825/766305500', 'heartbeat_back_addr': '172.21.15.60:6831/766305500', 'heartbeat_front_addr': '172.21.15.60:6828/766305500', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f36c332c-9497-47e8-b943-f6b6dcaebeb8', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6812', 'nonce': 1032956039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6816', 'nonce': 1032956039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 1032956039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6822', 'nonce': 1032956039}]}, 'public_addr': '172.21.15.60:6812/1032956039', 'cluster_addr': '172.21.15.60:6816/1032956039', 'heartbeat_back_addr': '172.21.15.60:6829/1032956039', 'heartbeat_front_addr': '172.21.15.60:6822/1032956039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 16238, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.183436+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.264080+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.253587+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.244429+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.369458+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.158089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.217295+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.332254+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.a', 'osds': [4, 2]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:09:10.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:09:10.636 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:10.554+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:10.637 INFO:teuthology.orchestra.run.smithi031.stdout:5215 read 0x193355a thru 0x1935ecf (0x2976 bytes) 2024-09-13T09:09:10.638 INFO:teuthology.orchestra.run.smithi031.stdout:5216 writesame 0x636e287 thru 0x63711cc data_size 0x3(0x2f46 bytes) 2024-09-13T09:09:10.638 INFO:teuthology.orchestra.run.smithi031.stdout:5217 read 0x5f10311 thru 0x5f1963a (0x932a bytes) 2024-09-13T09:09:10.638 INFO:teuthology.orchestra.run.smithi031.stdout:5218 write 0x1deffa8 thru 0x1df8b5a (0x8bb3 bytes) 2024-09-13T09:09:10.638 INFO:teuthology.orchestra.run.smithi031.stdout:5219 write 0x1902857 thru 0x1904996 (0x2140 bytes) 2024-09-13T09:09:10.638 INFO:teuthology.orchestra.run.smithi031.stdout:5221 write 0xb5991ce thru 0xb59df2e (0x4d61 bytes) 2024-09-13T09:09:10.649 INFO:teuthology.orchestra.run.smithi031.stdout:5222 trunc from 0xb59df2f to 0x6bac1c 2024-09-13T09:09:10.656 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:10.652+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:10.720 INFO:teuthology.orchestra.run.smithi031.stdout:5223 trunc from 0x6bac1c to 0xdc9b51e 2024-09-13T09:09:10.735 INFO:teuthology.orchestra.run.smithi031.stdout:5224 trunc from 0xdc9b51e to 0xb026621 2024-09-13T09:09:10.751 INFO:teuthology.orchestra.run.smithi031.stdout:5226 trunc from 0xb026621 to 0x943ba3c 2024-09-13T09:09:10.757 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:10.752+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:10.764 INFO:teuthology.orchestra.run.smithi031.stdout:5227 writesame 0x1a47f5e thru 0x1a55880 data_size 0x7(0xd923 bytes) 2024-09-13T09:09:10.767 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4388.0:147559 3.8 e114/103 3:123d49cc:::rbd_object_map.1124651f5f97:head v 114'414, mlcod=114'412)", 2024-09-13T09:09:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:10.766878+0000", 2024-09-13T09:09:10.767 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00039638199999999997, 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041519099999999998, 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:10.766878+0000", 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:10.766878+0000", 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:10.766877+0000", 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:10.766886+0000", 2024-09-13T09:09:10.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1730000000000001e-06 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:10.766903+0000", 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7017999999999999e-05 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:10.766911+0000", 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:10.766966+0000", 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5164000000000002e-05 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:10.766994+0000", 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7702e-05 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-13T09:09:10.771 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:10.778 INFO:teuthology.orchestra.run.smithi031.stdout:5228 read 0x4c4a1e7 thru 0x4c5848c (0xe2a6 bytes) 2024-09-13T09:09:10.778 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:10.779 INFO:teuthology.orchestra.run.smithi031.stdout:5231 trunc from 0x943ba3c to 0x9e3a146 2024-09-13T09:09:10.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:10.803 INFO:teuthology.orchestra.run.smithi031.stdout:5232 trunc from 0x9e3a146 to 0x182277f 2024-09-13T09:09:10.825 INFO:teuthology.orchestra.run.smithi031.stdout:5233 read 0xdb2d31 thru 0xdba18e (0x745e bytes) 2024-09-13T09:09:10.827 INFO:teuthology.orchestra.run.smithi031.stdout:5236 trunc from 0x182277f to 0x32f786b 2024-09-13T09:09:10.839 INFO:teuthology.orchestra.run.smithi031.stdout:5237 read 0x21ab9c3 thru 0x21b7634 (0xbc72 bytes) 2024-09-13T09:09:10.839 INFO:teuthology.orchestra.run.smithi031.stdout:5238 read 0xf8e795 thru 0xf9c0af (0xd91b bytes) 2024-09-13T09:09:10.839 INFO:teuthology.orchestra.run.smithi031.stdout:5239 read 0x195b8c5 thru 0x195f7cb (0x3f07 bytes) 2024-09-13T09:09:10.839 INFO:teuthology.orchestra.run.smithi031.stdout:5241 read 0x22af2e9 thru 0x22bc552 (0xd26a bytes) 2024-09-13T09:09:10.858 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:10.854+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:10.867 INFO:teuthology.orchestra.run.smithi031.stdout:5242 writesame 0xd57e3ab thru 0xd589a57 data_size 0x2f(0xb6ad bytes) 2024-09-13T09:09:10.887 INFO:teuthology.orchestra.run.smithi031.stdout:5243 trunc from 0xd589a58 to 0x136fc3f 2024-09-13T09:09:10.899 INFO:teuthology.orchestra.run.smithi031.stdout:5244 writesame 0xf6a216 thru 0xf6fc6c data_size 0xd(0x5a57 bytes) 2024-09-13T09:09:10.906 INFO:teuthology.orchestra.run.smithi031.stdout:5245 writesame 0x9d202d thru 0x9d6a47 data_size 0x1(0x4a1b bytes) 2024-09-13T09:09:10.923 INFO:teuthology.orchestra.run.smithi031.stdout:5246 read 0x79a5f6 thru 0x7a0bcb (0x65d6 bytes) 2024-09-13T09:09:10.924 INFO:teuthology.orchestra.run.smithi031.stdout:5247 read 0xad8789 thru 0xad8bb3 (0x42b bytes) 2024-09-13T09:09:10.925 INFO:teuthology.orchestra.run.smithi031.stdout:5248 read 0x2a7109 thru 0x2b60a2 (0xef9a bytes) 2024-09-13T09:09:10.928 INFO:teuthology.orchestra.run.smithi031.stdout:5249 read 0x5b47bf thru 0x5b9fd7 (0x5819 bytes) 2024-09-13T09:09:10.929 INFO:teuthology.orchestra.run.smithi031.stdout:5250 punch from 0x12c8b38 to 0x12d6489, (0xd951 bytes) 2024-09-13T09:09:10.929 INFO:teuthology.orchestra.run.smithi031.stdout:5251 read 0x1042419 thru 0x1050ecf (0xeab7 bytes) 2024-09-13T09:09:10.935 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:10.935 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:10.945 INFO:teuthology.orchestra.run.smithi031.stdout:5252 write 0x247e374 thru 0x248b0b2 (0xcd3f bytes) 2024-09-13T09:09:10.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:10.959 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:10.952+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:10.977 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:09:10.979 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:09:10.982 INFO:teuthology.orchestra.run.smithi031.stdout:5253 write 0xc3b4340 thru 0xc3ba10d (0x5dce bytes) 2024-09-13T09:09:10.990 INFO:teuthology.orchestra.run.smithi031.stdout:5254 read 0x59a1017 thru 0x59a7f72 (0x6f5c bytes) 2024-09-13T09:09:10.990 INFO:teuthology.orchestra.run.smithi031.stdout:5255 punch from 0xc66cb1 to 0xc7204d, (0xb39c bytes) 2024-09-13T09:09:10.990 INFO:teuthology.orchestra.run.smithi031.stdout:5257 write 0xb8a99ba thru 0xb8b4c6a (0xb2b1 bytes) 2024-09-13T09:09:10.998 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-13T09:09:10.037364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1501949713,"num_objects":1504,"num_object_clones":329,"num_object_copies":3008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1504,"num_whiteouts":575,"num_read":178675,"num_read_kb":602162,"num_write":19116,"num_write_kb":307423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1563,"num_bytes_recovered":165326340,"num_keys_recovered":348,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15121,"ondisk_log_size":15121,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":619096,"kb_used_data":335628,"kb_used_omap":9,"kb_used_meta":283382,"kb_avail":565611944,"statfs":{"total":579820584960,"available":579186630656,"internally_reserved":0,"allocated":343683072,"data_stored":370314737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":290183847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":105703391,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":9,"num_read":3301,"num_read_kb":12015,"num_write":463,"num_write_kb":5640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":6214525,"num_keys_recovered":33,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.956464"},"pg_stats":[{"pgid":"3.1f","version":"110'445","reported_seq":4866,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-13T09:09:07.027092+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:09:07.027092+0000","last_peered":"2024-09-13T09:09:07.027092+0000","last_clean":"2024-09-13T09:09:07.027092+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:09:07.027092+0000","last_undegraded":"2024-09-13T09:09:07.027092+0000","last_fullsized":"2024-09-13T09:09:07.027092+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":36529436,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":5446,"num_read_kb":16128,"num_write":581,"num_write_kb":4023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"113'676","reported_seq":2763,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T09:09:08.038724+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:09:08.038724+0000","last_peered":"2024-09-13T09:09:08.038724+0000","last_clean":"2024-09-13T09:09:08.038724+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:09:08.038724+0000","last_undegraded":"2024-09-13T09:09:08.038724+0000","last_fullsized":"2024-09-13T09:09:08.038724+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":45864879,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":4793,"num_read_kb":12408,"num_write":885,"num_write_kb":5148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"110'1306","reported_seq":4980,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.849132+0000","last_change":"2024-09-13T09:08:38.210944+0000","last_active":"2024-09-13T09:09:05.849132+0000","last_peered":"2024-09-13T09:09:05.849132+0000","last_clean":"2024-09-13T09:09:05.849132+0000","last_became_active":"2024-09-13T09:08:38.208191+0000","last_became_peered":"2024-09-13T09:08:38.208191+0000","last_unstale":"2024-09-13T09:09:05.849132+0000","last_undegraded":"2024-09-13T09:09:05.849132+0000","last_fullsized":"2024-09-13T09:09:05.849132+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:04.686300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":53392166,"num_objects":58,"num_object_clones":15,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":28,"num_read":8641,"num_read_kb":33475,"num_write":2170,"num_write_kb":16969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"111'419","reported_seq":3028,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T09:09:06.071364+0000","last_change":"2024-09-13T09:08:38.212078+0000","last_active":"2024-09-13T09:09:06.071364+0000","last_peered":"2024-09-13T09:09:06.071364+0000","last_clean":"2024-09-13T09:09:06.071364+0000","last_became_active":"2024-09-13T09:08:38.209451+0000","last_became_peered":"2024-09-13T09:08:38.209451+0000","last_unstale":"2024-09-13T09:09:06.071364+0000","last_undegraded":"2024-09-13T09:09:06.071364+0000","last_fullsized":"2024-09-13T09:09:06.071364+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:59.672566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":77031665,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":16,"num_read":4432,"num_read_kb":33241,"num_write":570,"num_write_kb":15118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"104'484","reported_seq":21111,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:03.998525+0000","last_change":"2024-09-13T09:08:38.212581+0000","last_active":"2024-09-13T09:09:03.998525+0000","last_peered":"2024-09-13T09:09:03.998525+0000","last_clean":"2024-09-13T09:09:03.998525+0000","last_became_active":"2024-09-13T09:08:38.210333+0000","last_became_peered":"2024-09-13T09:08:38.210333+0000","last_unstale":"2024-09-13T09:09:03.998525+0000","last_undegraded":"2024-09-13T09:09:03.998525+0000","last_fullsized":"2024-09-13T09:09:03.998525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:53.493070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":90976883,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":17,"num_read":22443,"num_read_kb":45299,"num_write":547,"num_write_kb":18207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"110'525","reported_seq":4889,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T09:09:06.133885+0000","last_change":"2024-09-13T09:08:09.864519+0000","last_active":"2024-09-13T09:09:06.133885+0000","last_peered":"2024-09-13T09:09:06.133885+0000","last_clean":"2024-09-13T09:09:06.133885+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:09:06.133885+0000","last_undegraded":"2024-09-13T09:09:06.133885+0000","last_fullsized":"2024-09-13T09:09:06.133885+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":60629886,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":6842,"num_read_kb":21905,"num_write":697,"num_write_kb":18892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"110'274","reported_seq":1202,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.298991+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:09:05.298991+0000","last_peered":"2024-09-13T09:09:05.298991+0000","last_clean":"2024-09-13T09:09:05.298991+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:09:05.298991+0000","last_undegraded":"2024-09-13T09:09:05.298991+0000","last_fullsized":"2024-09-13T09:09:05.298991+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":14084338,"num_objects":43,"num_object_clones":7,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":16,"num_read":1150,"num_read_kb":5529,"num_write":259,"num_write_kb":6501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"111'530","reported_seq":2707,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T09:09:08.100444+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:09:08.100444+0000","last_peered":"2024-09-13T09:09:08.100444+0000","last_clean":"2024-09-13T09:09:08.100444+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:09:08.100444+0000","last_undegraded":"2024-09-13T09:09:08.100444+0000","last_fullsized":"2024-09-13T09:09:08.100444+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":45848292,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":19,"num_read":3826,"num_read_kb":12965,"num_write":636,"num_write_kb":9753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"109'392","reported_seq":4548,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-13T09:09:07.027220+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:09:07.027220+0000","last_peered":"2024-09-13T09:09:07.027220+0000","last_clean":"2024-09-13T09:09:07.027220+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:09:07.027220+0000","last_undegraded":"2024-09-13T09:09:07.027220+0000","last_fullsized":"2024-09-13T09:09:07.027220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":53355709,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":5133,"num_read_kb":28859,"num_write":411,"num_write_kb":17463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"110'606","reported_seq":1487,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.990182+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:09:05.990182+0000","last_peered":"2024-09-13T09:09:05.990182+0000","last_clean":"2024-09-13T09:09:05.990182+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:09:05.990182+0000","last_undegraded":"2024-09-13T09:09:05.990182+0000","last_fullsized":"2024-09-13T09:09:05.990182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":35472392,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":2631,"num_read_kb":12531,"num_write":887,"num_write_kb":9354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"104'484","reported_seq":3270,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:04.297335+0000","last_change":"2024-09-13T09:08:38.222834+0000","last_active":"2024-09-13T09:09:04.297335+0000","last_peered":"2024-09-13T09:09:04.297335+0000","last_clean":"2024-09-13T09:09:04.297335+0000","last_became_active":"2024-09-13T09:08:38.221484+0000","last_became_peered":"2024-09-13T09:08:38.221484+0000","last_unstale":"2024-09-13T09:09:04.297335+0000","last_undegraded":"2024-09-13T09:09:04.297335+0000","last_fullsized":"2024-09-13T09:09:04.297335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:04.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":92172088,"num_objects":52,"num_object_clones":16,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":16,"num_read":3882,"num_read_kb":22497,"num_write":507,"num_write_kb":14518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"109'557","reported_seq":18491,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.637021+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:09:05.637021+0000","last_peered":"2024-09-13T09:09:05.637021+0000","last_clean":"2024-09-13T09:09:05.637021+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:09:05.637021+0000","last_undegraded":"2024-09-13T09:09:05.637021+0000","last_fullsized":"2024-09-13T09:09:05.637021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":46468143,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":16,"num_read":20303,"num_read_kb":32085,"num_write":519,"num_write_kb":9556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":114,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.057938+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:09:05.057938+0000","last_peered":"2024-09-13T09:09:05.057938+0000","last_clean":"2024-09-13T09:09:05.057938+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:09:05.057938+0000","last_undegraded":"2024-09-13T09:09:05.057938+0000","last_fullsized":"2024-09-13T09:09:05.057938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'523","reported_seq":4992,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:04.829387+0000","last_change":"2024-09-13T09:08:57.963388+0000","last_active":"2024-09-13T09:09:04.829387+0000","last_peered":"2024-09-13T09:09:04.829387+0000","last_clean":"2024-09-13T09:09:04.829387+0000","last_became_active":"2024-09-13T09:08:57.961197+0000","last_became_peered":"2024-09-13T09:08:57.961197+0000","last_unstale":"2024-09-13T09:09:04.829387+0000","last_undegraded":"2024-09-13T09:09:04.829387+0000","last_fullsized":"2024-09-13T09:09:04.829387+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:00.864515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":49677175,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":5701,"num_read_kb":20229,"num_write":685,"num_write_kb":6435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":6759140,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":113,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:03.998498+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:09:03.998498+0000","last_peered":"2024-09-13T09:09:03.998498+0000","last_clean":"2024-09-13T09:09:03.998498+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:09:03.998498+0000","last_undegraded":"2024-09-13T09:09:03.998498+0000","last_fullsized":"2024-09-13T09:09:03.998498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"109'310","reported_seq":1960,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:04.669439+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:09:04.669439+0000","last_peered":"2024-09-13T09:09:04.669439+0000","last_clean":"2024-09-13T09:09:04.669439+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:09:04.669439+0000","last_undegraded":"2024-09-13T09:09:04.669439+0000","last_fullsized":"2024-09-13T09:09:04.669439+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":23933200,"num_objects":37,"num_object_clones":5,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":2412,"num_read_kb":13485,"num_write":296,"num_write_kb":5018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":106,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.057973+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:09:05.057973+0000","last_peered":"2024-09-13T09:09:05.057973+0000","last_clean":"2024-09-13T09:09:05.057973+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:09:05.057973+0000","last_undegraded":"2024-09-13T09:09:05.057973+0000","last_fullsized":"2024-09-13T09:09:05.057973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"109'398","reported_seq":5186,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:04.188819+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:09:04.188819+0000","last_peered":"2024-09-13T09:09:04.188819+0000","last_clean":"2024-09-13T09:09:04.188819+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:09:04.188819+0000","last_undegraded":"2024-09-13T09:09:04.188819+0000","last_fullsized":"2024-09-13T09:09:04.188819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":47154773,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":5703,"num_read_kb":19950,"num_write":544,"num_write_kb":4841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T09:09:08.037983+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:09:08.037983+0000","last_peered":"2024-09-13T09:09:08.037983+0000","last_clean":"2024-09-13T09:09:08.037983+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:09:08.037983+0000","last_undegraded":"2024-09-13T09:09:08.037983+0000","last_fullsized":"2024-09-13T09:09:08.037983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"110'533","reported_seq":2748,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.850268+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:09:05.850268+0000","last_peered":"2024-09-13T09:09:05.850268+0000","last_clean":"2024-09-13T09:09:05.850268+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:09:05.850268+0000","last_undegraded":"2024-09-13T09:09:05.850268+0000","last_fullsized":"2024-09-13T09:09:05.850268+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":40574787,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":3783,"num_read_kb":16406,"num_write":743,"num_write_kb":10138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"110'337","reported_seq":2451,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.990943+0000","last_change":"2024-09-13T09:08:38.211675+0000","last_active":"2024-09-13T09:09:05.990943+0000","last_peered":"2024-09-13T09:09:05.990943+0000","last_clean":"2024-09-13T09:09:05.990943+0000","last_became_active":"2024-09-13T09:08:38.209914+0000","last_became_peered":"2024-09-13T09:08:38.209914+0000","last_unstale":"2024-09-13T09:09:05.990943+0000","last_undegraded":"2024-09-13T09:09:05.990943+0000","last_fullsized":"2024-09-13T09:09:05.990943+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:19.947594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":78970947,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":2634,"num_read_kb":14598,"num_write":363,"num_write_kb":5329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":918446,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":108,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.057519+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:09:05.057519+0000","last_peered":"2024-09-13T09:09:05.057519+0000","last_clean":"2024-09-13T09:09:05.057519+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:09:05.057519+0000","last_undegraded":"2024-09-13T09:09:05.057519+0000","last_fullsized":"2024-09-13T09:09:05.057519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"113'360","reported_seq":2280,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T09:09:08.238905+0000","last_change":"2024-09-13T09:08:57.959945+0000","last_active":"2024-09-13T09:09:08.238905+0000","last_peered":"2024-09-13T09:09:08.238905+0000","last_clean":"2024-09-13T09:09:08.238905+0000","last_became_active":"2024-09-13T09:08:57.957875+0000","last_became_peered":"2024-09-13T09:08:57.957875+0000","last_unstale":"2024-09-13T09:09:08.238905+0000","last_undegraded":"2024-09-13T09:09:08.238905+0000","last_fullsized":"2024-09-13T09:09:08.238905+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.313991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":22427243,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":2374,"num_read_kb":12401,"num_write":402,"num_write_kb":7178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3111448,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-13T09:09:07.026579+0000","last_change":"2024-09-13T09:08:54.932713+0000","last_active":"2024-09-13T09:09:07.026579+0000","last_peered":"2024-09-13T09:09:07.026579+0000","last_clean":"2024-09-13T09:09:07.026579+0000","last_became_active":"2024-09-13T09:08:54.931779+0000","last_became_peered":"2024-09-13T09:08:54.931779+0000","last_unstale":"2024-09-13T09:09:07.026579+0000","last_undegraded":"2024-09-13T09:09:07.026579+0000","last_fullsized":"2024-09-13T09:09:07.026579+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:42:37.917578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"107'308","reported_seq":1684,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:04.017170+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:09:04.017170+0000","last_peered":"2024-09-13T09:09:04.017170+0000","last_clean":"2024-09-13T09:09:04.017170+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:09:04.017170+0000","last_undegraded":"2024-09-13T09:09:04.017170+0000","last_fullsized":"2024-09-13T09:09:04.017170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":15265327,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":1675,"num_read_kb":12812,"num_write":322,"num_write_kb":6675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":107,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T09:09:06.017541+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:09:06.017541+0000","last_peered":"2024-09-13T09:09:06.017541+0000","last_clean":"2024-09-13T09:09:06.017541+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:09:06.017541+0000","last_undegraded":"2024-09-13T09:09:06.017541+0000","last_fullsized":"2024-09-13T09:09:06.017541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"107'321","reported_seq":1924,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:03.998092+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:09:03.998092+0000","last_peered":"2024-09-13T09:09:03.998092+0000","last_clean":"2024-09-13T09:09:03.998092+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:09:03.998092+0000","last_undegraded":"2024-09-13T09:09:03.998092+0000","last_fullsized":"2024-09-13T09:09:03.998092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":16147545,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":2134,"num_read_kb":9749,"num_write":354,"num_write_kb":2819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":107,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.057723+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:09:05.057723+0000","last_peered":"2024-09-13T09:09:05.057723+0000","last_clean":"2024-09-13T09:09:05.057723+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:09:05.057723+0000","last_undegraded":"2024-09-13T09:09:05.057723+0000","last_fullsized":"2024-09-13T09:09:05.057723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":236,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.014968+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:09:05.014968+0000","last_peered":"2024-09-13T09:09:05.014968+0000","last_clean":"2024-09-13T09:09:05.014968+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:09:05.014968+0000","last_undegraded":"2024-09-13T09:09:05.014968+0000","last_fullsized":"2024-09-13T09:09:05.014968+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"110'382","reported_seq":4772,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.051617+0000","last_change":"2024-09-13T09:08:57.966773+0000","last_active":"2024-09-13T09:09:05.051617+0000","last_peered":"2024-09-13T09:09:05.051617+0000","last_clean":"2024-09-13T09:09:05.051617+0000","last_became_active":"2024-09-13T09:08:57.964033+0000","last_became_peered":"2024-09-13T09:08:57.964033+0000","last_unstale":"2024-09-13T09:09:05.051617+0000","last_undegraded":"2024-09-13T09:09:05.051617+0000","last_fullsized":"2024-09-13T09:09:05.051617+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:54.967175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":56458063,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":4507,"num_read_kb":20986,"num_write":517,"num_write_kb":10353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":4982043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"109'438","reported_seq":2990,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.714715+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:09:05.714715+0000","last_peered":"2024-09-13T09:09:05.714715+0000","last_clean":"2024-09-13T09:09:05.714715+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:09:05.714715+0000","last_undegraded":"2024-09-13T09:09:05.714715+0000","last_fullsized":"2024-09-13T09:09:05.714715+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":47878367,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":4245,"num_read_kb":16001,"num_write":548,"num_write_kb":8690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"110'448","reported_seq":2306,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.011237+0000","last_change":"2024-09-13T09:08:21.089403+0000","last_active":"2024-09-13T09:09:05.011237+0000","last_peered":"2024-09-13T09:09:05.011237+0000","last_clean":"2024-09-13T09:09:05.011237+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:09:05.011237+0000","last_undegraded":"2024-09-13T09:09:05.011237+0000","last_fullsized":"2024-09-13T09:09:05.011237+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":82258226,"num_objects":37,"num_object_clones":11,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":14,"num_read":3376,"num_read_kb":21127,"num_write":438,"num_write_kb":16855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"109'307","reported_seq":2871,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:04.817524+0000","last_change":"2024-09-13T09:08:57.963838+0000","last_active":"2024-09-13T09:09:04.817524+0000","last_peered":"2024-09-13T09:09:04.817524+0000","last_clean":"2024-09-13T09:09:04.817524+0000","last_became_active":"2024-09-13T09:08:57.961707+0000","last_became_peered":"2024-09-13T09:08:57.961707+0000","last_unstale":"2024-09-13T09:09:04.817524+0000","last_undegraded":"2024-09-13T09:09:04.817524+0000","last_fullsized":"2024-09-13T09:09:04.817524+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:58.156711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":37042484,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":2914,"num_read_kb":26293,"num_write":337,"num_write_kb":16107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":8453655,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"111'240","reported_seq":16476,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T09:09:06.147095+0000","last_change":"2024-09-13T09:08:57.961063+0000","last_active":"2024-09-13T09:09:06.147095+0000","last_peered":"2024-09-13T09:09:06.147095+0000","last_clean":"2024-09-13T09:09:06.147095+0000","last_became_active":"2024-09-13T09:08:57.959990+0000","last_became_peered":"2024-09-13T09:08:57.959990+0000","last_unstale":"2024-09-13T09:09:06.147095+0000","last_undegraded":"2024-09-13T09:09:06.147095+0000","last_fullsized":"2024-09-13T09:09:06.147095+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:21.370554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":49639484,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":16205,"num_read_kb":26214,"num_write":271,"num_write_kb":14066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":26451574,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"109'293","reported_seq":2590,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.768160+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:09:05.768160+0000","last_peered":"2024-09-13T09:09:05.768160+0000","last_clean":"2024-09-13T09:09:05.768160+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:09:05.768160+0000","last_undegraded":"2024-09-13T09:09:05.768160+0000","last_fullsized":"2024-09-13T09:09:05.768160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":31542849,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":2791,"num_read_kb":8694,"num_write":289,"num_write_kb":4566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"107'677","reported_seq":6206,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.338512+0000","last_change":"2024-09-13T09:08:38.205741+0000","last_active":"2024-09-13T09:09:05.338512+0000","last_peered":"2024-09-13T09:09:05.338512+0000","last_clean":"2024-09-13T09:09:05.338512+0000","last_became_active":"2024-09-13T09:08:38.205194+0000","last_became_peered":"2024-09-13T09:08:38.205194+0000","last_unstale":"2024-09-13T09:09:05.338512+0000","last_undegraded":"2024-09-13T09:09:05.338512+0000","last_fullsized":"2024-09-13T09:09:05.338512+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:14.407799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":49211509,"num_objects":52,"num_object_clones":14,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":18,"num_read":8281,"num_read_kb":21727,"num_write":1063,"num_write_kb":4497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"109'314","reported_seq":4013,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T09:09:06.031827+0000","last_change":"2024-09-13T09:09:04.008635+0000","last_active":"2024-09-13T09:09:06.031827+0000","last_peered":"2024-09-13T09:09:06.031827+0000","last_clean":"2024-09-13T09:09:06.031827+0000","last_became_active":"2024-09-13T09:09:04.006127+0000","last_became_peered":"2024-09-13T09:09:04.006127+0000","last_unstale":"2024-09-13T09:09:06.031827+0000","last_undegraded":"2024-09-13T09:09:06.031827+0000","last_fullsized":"2024-09-13T09:09:06.031827+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:37.417526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":42582629,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":4198,"num_read_kb":14236,"num_write":300,"num_write_kb":5693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":237,"num_bytes_recovered":6892313,"num_keys_recovered":55,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"110'373","reported_seq":2334,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.763506+0000","last_change":"2024-09-13T09:08:09.863895+0000","last_active":"2024-09-13T09:09:05.763506+0000","last_peered":"2024-09-13T09:09:05.763506+0000","last_clean":"2024-09-13T09:09:05.763506+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:09:05.763506+0000","last_undegraded":"2024-09-13T09:09:05.763506+0000","last_fullsized":"2024-09-13T09:09:05.763506+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":58050848,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":2566,"num_read_kb":16873,"num_write":371,"num_write_kb":9135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"109'761","reported_seq":3263,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.057545+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:09:05.057545+0000","last_peered":"2024-09-13T09:09:05.057545+0000","last_clean":"2024-09-13T09:09:05.057545+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:09:05.057545+0000","last_undegraded":"2024-09-13T09:09:05.057545+0000","last_fullsized":"2024-09-13T09:09:05.057545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":39740366,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":13,"num_read":5610,"num_read_kb":10433,"num_write":1124,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"107'548","reported_seq":3626,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T09:09:04.303907+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:09:04.303907+0000","last_peered":"2024-09-13T09:09:04.303907+0000","last_clean":"2024-09-13T09:09:04.303907+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:09:04.303907+0000","last_undegraded":"2024-09-13T09:09:04.303907+0000","last_fullsized":"2024-09-13T09:09:04.303907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":39760839,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":13,"num_read":5089,"num_read_kb":12734,"num_write":833,"num_write_kb":9917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"107'432","reported_seq":3043,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T09:09:05.057620+0000","last_change":"2024-09-13T09:09:04.006383+0000","last_active":"2024-09-13T09:09:05.057620+0000","last_peered":"2024-09-13T09:09:05.057620+0000","last_clean":"2024-09-13T09:09:05.057620+0000","last_became_active":"2024-09-13T09:09:04.005219+0000","last_became_peered":"2024-09-13T09:09:04.005219+0000","last_unstale":"2024-09-13T09:09:05.057620+0000","last_undegraded":"2024-09-13T09:09:05.057620+0000","last_fullsized":"2024-09-13T09:09:05.057620+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:56.239198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":21216788,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":2839,"num_read_kb":10192,"num_write":486,"num_write_kb":6469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1501359326,"num_objects":1500,"num_object_clones":329,"num_object_copies":3000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1500,"num_whiteouts":575,"num_read":178559,"num_read_kb":602062,"num_write":18955,"num_write_kb":305005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1561,"num_bytes_recovered":164735972,"num_keys_recovered":348,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376410112,"data_stored":412507238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15001,"ondisk_log_size":15001,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509469,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145084,"kb_used_data":73776,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94226756,"statfs":{"total":96636764160,"available":96488198144,"internally_reserved":0,"allocated":75546624,"data_stored":84424822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.115,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.27200000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.121,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.41399999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.071,"5min":1.444,"15min":1.444},"last":0.441},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59399999999999997,"15min":0.57199999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85199999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2989999999999999,"5min":1.343,"15min":1.343},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.873,"15min":0.86499999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.92200000000000004,"15min":0.90600000000000003},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.92500000000000004,"15min":0.92100000000000004},"min":{"1min":0.52600000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.79600000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.69599999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104336,"kb_used_data":51080,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94267504,"statfs":{"total":96636764160,"available":96529924096,"internally_reserved":0,"allocated":52305920,"data_stored":53923652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73999999999999999,"15min":0.751},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80600000000000005,"15min":0.79900000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.502},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.88100000000000001,"15min":0.91400000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.0109999999999999,"15min":1.0389999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.95499999999999996},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.492,"5min":1.651,"15min":1.651},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.98299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3049999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.107,"15min":1.131},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5840000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.95199999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.873},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":0.80700000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95832,"kb_used_data":52424,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94276008,"statfs":{"total":96636764160,"available":96538632192,"internally_reserved":0,"allocated":53682176,"data_stored":58505422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.625,"15min":0.64400000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.995,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.036,"5min":1.214,"15min":1.214},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96899999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.407,"15min":1.407},"last":0.496}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.121,"5min":1.415,"15min":1.415},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.92200000000000004,"15min":0.94199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2270000000000001,"5min":1.448,"15min":1.448},"last":0.75}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96599999999999997,"15min":0.98599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.96299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2010000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.47399999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.353,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.878,"5min":1.0309999999999999,"15min":1.0569999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3460000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":0.82999999999999996}]}]},{"osd":0,"up_from":0,"seq":64424509470,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65356,"kb_used_data":34244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94306484,"statfs":{"total":96636764160,"available":96569839616,"internally_reserved":0,"allocated":35065856,"data_stored":39280066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77700000000000002,"15min":0.76400000000000001},"min":{"1min":0.47499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1899999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.82199999999999995,"15min":0.79700000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.92900000000000005,"15min":0.92800000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":0.94899999999999995,"15min":0.90100000000000002},"min":{"1min":0.53700000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":8.3499999999999996,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.206,"5min":0.98199999999999998,"15min":0.94399999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":8.2989999999999995,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.88}]},{"osd":3,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.95099999999999996,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.472,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.92300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.218,"5min":1.7,"15min":1.7},"last":1.0169999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.085,"15min":1.093},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0700000000000001,"15min":1.0720000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1319999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.242},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1080000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97500,"kb_used_data":66380,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94274340,"statfs":{"total":96636764160,"available":96536924160,"internally_reserved":0,"allocated":67973120,"data_stored":73797580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.73999999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.68300000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.85599999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.88900000000000001,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":0.49299999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.008,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:08:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.55100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":110988,"kb_used_data":57724,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94260852,"statfs":{"total":96636764160,"available":96523112448,"internally_reserved":0,"allocated":59109376,"data_stored":60383195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":1.1200000000000001,"15min":1.179},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0489999999999999,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.085,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.218,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.64600000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.1080000000000001,"15min":1.159},"min":{"1min":0.54900000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.151,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.0089999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4910000000000001,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.96999999999999997,"15min":0.97799999999999998},"min":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.585},"last":0.92500000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0169999999999999,"15min":1.0309999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.357,"15min":1.415},"min":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3799999999999999,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":0.85199999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.012,"5min":1.0800000000000001,"15min":1.091},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4710000000000001,"5min":1.899,"15min":1.899},"last":0.97099999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.82699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6295552,"data_stored":7615357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51773440,"data_stored":57302232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51044352,"data_stored":53351670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":74530816,"data_stored":84325176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":66686976,"data_stored":73197140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57180160,"data_stored":59168082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":33771520,"data_stored":38679569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":35127296,"data_stored":38868012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:09:11.001 INFO:tasks.thrashosds.thrasher:Setting 3.f to [1, 3, 0, 5] 2024-09-13T09:09:11.001 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.f', '1', '3', '0', '5'] 2024-09-13T09:09:11.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.f 1 3 0 5 2024-09-13T09:09:11.002 INFO:teuthology.orchestra.run.smithi031.stdout:5258 trunc from 0xc3ba10e to 0xb061583 2024-09-13T09:09:11.022 INFO:teuthology.orchestra.run.smithi031.stdout:5259 read 0x37f16cd thru 0x37facb0 (0x95e4 bytes) 2024-09-13T09:09:11.022 INFO:teuthology.orchestra.run.smithi031.stdout:5261 write 0x8a1617e thru 0x8a25a5b (0xf8de bytes) 2024-09-13T09:09:11.033 INFO:teuthology.orchestra.run.smithi031.stdout:5262 write 0xabb958f thru 0xabc007d (0x6aef bytes) 2024-09-13T09:09:11.040 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.041 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.041 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.047 INFO:teuthology.orchestra.run.smithi031.stdout:5269 trunc from 0xb061583 to 0x64f461d 2024-09-13T09:09:11.060 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:11.052+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:11.068 INFO:teuthology.orchestra.run.smithi031.stdout:5271 read 0x481ca1 thru 0x4824a0 (0x800 bytes) 2024-09-13T09:09:11.069 INFO:teuthology.orchestra.run.smithi031.stdout:5272 read 0x3a46e25 thru 0x3a54164 (0xd340 bytes) 2024-09-13T09:09:11.069 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.070 INFO:teuthology.orchestra.run.smithi031.stdout:5275 writesame 0x12a834 thru 0x12e4b2 data_size 0x1(0x3c7f bytes) 2024-09-13T09:09:11.114 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774330093000003, 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774313320000005, 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774276014999998, 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:09:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774256300999994, 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:09:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774239976000004, 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774224110000006, 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774208325000004, 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774189254000007, 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:09:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:09:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774172707000005, 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:09:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774145107999999, 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:09:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774127866000001, 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:09:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774097423000001, 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:09:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774078508000002, 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:09:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:09:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:09:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774060719999994, 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:09:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:09:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:09:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774035495999996, 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:09:11.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:09:11.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.774017291999996, 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.188 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:11.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.773996973999999, 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:09:11.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:09:11.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.773978425999999, 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:09:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.773961151999998, 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:09:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:09:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:09:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 95.773943697999997, 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:09:11.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.213 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:11.214 INFO:teuthology.orchestra.run.smithi031.stdout:5276 write 0xeb61e69 thru 0xeb697e5 (0x797d bytes) 2024-09-13T09:09:11.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:11.216 INFO:teuthology.orchestra.run.smithi031.stdout:5277 read 0x8a33a9a thru 0x8a3a591 (0x6af8 bytes) 2024-09-13T09:09:11.216 INFO:teuthology.orchestra.run.smithi031.stdout:5278 punch from 0x83f936f to 0x83fbc72, (0x2903 bytes) 2024-09-13T09:09:11.216 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.216 INFO:teuthology.orchestra.run.smithi031.stdout:5281 write 0xc7573ef thru 0xc760a62 (0x9674 bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5282 punch from 0x14f342a to 0x14f4110, (0xce6 bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5283 read 0xa0f38fc thru 0xa0fe9a5 (0xb0aa bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5284 read 0xbc139ce thru 0xbc2376a (0xfd9d bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5285 read 0xd11ce2a thru 0xd122cca (0x5ea1 bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5287 read 0x10dfb68 thru 0x10e2660 (0x2af9 bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5290 write 0x172b49b thru 0x1738358 (0xcebe bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5291 write 0xdac871f thru 0xdad2af1 (0xa3d3 bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5292 write 0x452858 thru 0x45b95a (0x9103 bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5293 read 0x83d200 thru 0x84ad67 (0xdb68 bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5295 read 0xca19a12 thru 0xca1be69 (0x2458 bytes) 2024-09-13T09:09:11.217 INFO:teuthology.orchestra.run.smithi031.stdout:5296 write 0x6f16a8a thru 0x6f18bba (0x2131 bytes) 2024-09-13T09:09:11.218 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:11.210+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:11.218 INFO:teuthology.orchestra.run.smithi031.stdout:5297 read 0x5dcad5b thru 0x5dd23c0 (0x7666 bytes) 2024-09-13T09:09:11.218 INFO:teuthology.orchestra.run.smithi031.stdout:5298 read 0x5d98a6c thru 0x5da22a5 (0x983a bytes) 2024-09-13T09:09:11.218 INFO:teuthology.orchestra.run.smithi031.stdout:5299 writesame 0xdd3cf8b thru 0xdd46e7e data_size 0x6(0x9ef4 bytes) 2024-09-13T09:09:11.227 INFO:teuthology.orchestra.run.smithi031.stdout:5301 read 0x181fbb8 thru 0x1828871 (0x8cba bytes) 2024-09-13T09:09:11.229 INFO:teuthology.orchestra.run.smithi031.stdout:5302 writesame 0x1586d thru 0x1e5a5 data_size 0x67(0x8d39 bytes) 2024-09-13T09:09:11.233 INFO:teuthology.orchestra.run.smithi031.stdout:5303 trunc from 0xeb697e6 to 0xe3cff07 2024-09-13T09:09:11.252 INFO:teuthology.orchestra.run.smithi031.stdout:5304 write 0xaeafa7d thru 0xaeb8c10 (0x9194 bytes) 2024-09-13T09:09:11.262 INFO:teuthology.orchestra.run.smithi031.stdout:5305 read 0xe1d1a5c thru 0xe1d4d74 (0x3319 bytes) 2024-09-13T09:09:11.262 INFO:teuthology.orchestra.run.smithi031.stdout:5306 read 0xc16bd51 thru 0xc17a3a5 (0xe655 bytes) 2024-09-13T09:09:11.275 INFO:teuthology.orchestra.run.smithi031.stdout:5307 trunc from 0xe3cff07 to 0x5106657 2024-09-13T09:09:11.299 INFO:teuthology.orchestra.run.smithi031.stdout:5309 read 0x456b056 thru 0x456bcd6 (0xc81 bytes) 2024-09-13T09:09:11.299 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.299 INFO:teuthology.orchestra.run.smithi031.stdout:5311 writesame 0x26bc870 thru 0x26c75e5 data_size 0x9(0xad76 bytes) 2024-09-13T09:09:11.315 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:11.310+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:11.318 INFO:teuthology.orchestra.run.smithi031.stdout:5315 trunc from 0x5106657 to 0xcc66184 2024-09-13T09:09:11.331 INFO:teuthology.orchestra.run.smithi031.stdout:5316 trunc from 0xcc66184 to 0x9eb8a79 2024-09-13T09:09:11.350 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:11.350 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:11.355 INFO:teuthology.orchestra.run.smithi031.stdout:5317 write 0xb8e55f0 thru 0xb8ec2dd (0x6cee bytes) 2024-09-13T09:09:11.363 INFO:teuthology.orchestra.run.smithi031.stdout:5322 punch from 0x947e077 to 0x9486b5d, (0x8ae6 bytes) 2024-09-13T09:09:11.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:11.366 INFO:teuthology.orchestra.run.smithi031.stdout:5324 trunc from 0xb8ec2de to 0x937a988 2024-09-13T09:09:11.385 INFO:teuthology.orchestra.run.smithi031.stdout:5325 read 0x67a9454 thru 0x67b421a (0xadc7 bytes) 2024-09-13T09:09:11.397 INFO:teuthology.orchestra.run.smithi031.stdout:5326 write 0xa1127dd thru 0xa121d31 (0xf555 bytes) 2024-09-13T09:09:11.407 INFO:teuthology.orchestra.run.smithi031.stdout:5327 write 0x1d2b1cf thru 0x1d34782 (0x95b4 bytes) 2024-09-13T09:09:11.415 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:11.408+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:11.418 INFO:teuthology.orchestra.run.smithi031.stdout:5328 trunc from 0xa121d32 to 0xcb2518b 2024-09-13T09:09:11.429 INFO:teuthology.orchestra.run.smithi031.stdout:5329 write 0xb9c429 thru 0xba2306 (0x5ede bytes) 2024-09-13T09:09:11.437 INFO:teuthology.orchestra.run.smithi031.stdout:5332 read 0x5336f3d thru 0x533f931 (0x89f5 bytes) 2024-09-13T09:09:11.437 INFO:teuthology.orchestra.run.smithi031.stdout:5333 read 0x9b1cba5 thru 0x9b2b5ef (0xea4b bytes) 2024-09-13T09:09:11.437 INFO:teuthology.orchestra.run.smithi031.stdout:5334 read 0x5caca95 thru 0x5cb143e (0x49aa bytes) 2024-09-13T09:09:11.437 INFO:teuthology.orchestra.run.smithi031.stdout:5338 writesame 0x15f9221 thru 0x1604db2 data_size 0x35(0xbb92 bytes) 2024-09-13T09:09:11.444 INFO:teuthology.orchestra.run.smithi031.stdout:5339 read 0x2dddd3b thru 0x2de7ea9 (0xa16f bytes) 2024-09-13T09:09:11.444 INFO:teuthology.orchestra.run.smithi031.stdout:5340 read 0xa90e6e9 thru 0xa91cd65 (0xe67d bytes) 2024-09-13T09:09:11.444 INFO:teuthology.orchestra.run.smithi031.stdout:5341 read 0x8f03285 thru 0x8f127a3 (0xf51f bytes) 2024-09-13T09:09:11.444 INFO:teuthology.orchestra.run.smithi031.stdout:5342 writesame 0x6c1dd92 thru 0x6c2bcde data_size 0x6f(0xdf4d bytes) 2024-09-13T09:09:11.455 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.455 INFO:teuthology.orchestra.run.smithi031.stdout:5345 punch from 0x3450a44 to 0x34599a1, (0x8f5d bytes) 2024-09-13T09:09:11.455 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.470 INFO:teuthology.orchestra.run.smithi031.stdout:5347 writesame 0xea15bf4 thru 0xea15d12 data_size 0x1(0x11f bytes) 2024-09-13T09:09:11.482 INFO:teuthology.orchestra.run.smithi031.stdout:5348 trunc from 0xea15d13 to 0x426f4bc 2024-09-13T09:09:11.483 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:11.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:11.483 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:11.483 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:11.483 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:11.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:11.505 INFO:teuthology.orchestra.run.smithi031.stdout:5351 punch from 0x3dc33f2 to 0x3dccf95, (0x9ba3 bytes) 2024-09-13T09:09:11.516 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:11.508+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:11.517 INFO:teuthology.orchestra.run.smithi031.stdout:5352 writesame 0x4e23562 thru 0x4e2fd08 data_size 0x1b(0xc7a7 bytes) 2024-09-13T09:09:11.530 INFO:teuthology.orchestra.run.smithi031.stdout:5353 writesame 0x6c8504f thru 0x6c8f913 data_size 0x1(0xa8c5 bytes) 2024-09-13T09:09:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:5355 writesame 0x83a9a55 thru 0x83ad7b8 data_size 0x2(0x3d64 bytes) 2024-09-13T09:09:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.558 INFO:teuthology.orchestra.run.smithi031.stdout:5357 write 0x955f92e thru 0x9568d61 (0x9434 bytes) 2024-09-13T09:09:11.568 INFO:teuthology.orchestra.run.smithi031.stdout:5358 punch from 0x8496961 to 0x84a5cea, (0xf389 bytes) 2024-09-13T09:09:11.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.569 INFO:teuthology.orchestra.run.smithi031.stdout:5364 write 0x92a8698 thru 0x92ac5c2 (0x3f2b bytes) 2024-09-13T09:09:11.572 INFO:teuthology.orchestra.run.smithi031.stdout:5365 read 0x772ce8 thru 0x778ad9 (0x5df2 bytes) 2024-09-13T09:09:11.573 INFO:teuthology.orchestra.run.smithi031.stdout:5366 punch from 0x6a0c04 to 0x6adc0b, (0xd007 bytes) 2024-09-13T09:09:11.573 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.573 INFO:teuthology.orchestra.run.smithi031.stdout:5371 punch from 0x7f7846 to 0x8067db, (0xef95 bytes) 2024-09-13T09:09:11.574 INFO:teuthology.orchestra.run.smithi031.stdout:5372 punch from 0x485d082 to 0x485d791, (0x70f bytes) 2024-09-13T09:09:11.574 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.574 INFO:teuthology.orchestra.run.smithi031.stdout:5374 write 0x403887d thru 0x40436b4 (0xae38 bytes) 2024-09-13T09:09:11.581 INFO:teuthology.orchestra.run.smithi031.stdout:5375 write 0x8904ce thru 0x89e45f (0xdf92 bytes) 2024-09-13T09:09:11.585 INFO:teuthology.orchestra.run.smithi031.stdout:5376 read 0x312135e thru 0x31270ff (0x5da2 bytes) 2024-09-13T09:09:11.585 INFO:teuthology.orchestra.run.smithi031.stdout:5377 read 0xff04fe thru 0xffe0f7 (0xdbfa bytes) 2024-09-13T09:09:11.585 INFO:teuthology.orchestra.run.smithi031.stdout:5378 read 0x954fa09 thru 0x95508d1 (0xec9 bytes) 2024-09-13T09:09:11.585 INFO:teuthology.orchestra.run.smithi031.stdout:5379 writesame 0x3b88b2b thru 0x3b98472 data_size 0xc(0xf948 bytes) 2024-09-13T09:09:11.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:11.601 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.601 INFO:teuthology.orchestra.run.smithi031.stdout:5384 punch from 0x800d6b3 to 0x8011901, (0x424e bytes) 2024-09-13T09:09:11.601 INFO:teuthology.orchestra.run.smithi031.stdout:5385 punch from 0x5e5e063 to 0x5e64963, (0x6900 bytes) 2024-09-13T09:09:11.601 INFO:teuthology.orchestra.run.smithi031.stdout:5386 write 0x222bdb3 thru 0x223b87c (0xfaca bytes) 2024-09-13T09:09:11.610 INFO:teuthology.orchestra.run.smithi031.stdout:5387 writesame 0x7be480a thru 0x7be8e7c data_size 0x1(0x4673 bytes) 2024-09-13T09:09:11.616 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:11.614+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:11.626 INFO:teuthology.orchestra.run.smithi031.stdout:5388 read 0x13914e0 thru 0x139746c (0x5f8d bytes) 2024-09-13T09:09:11.627 INFO:teuthology.orchestra.run.smithi031.stdout:5391 trunc from 0x9568d62 to 0x8bf4b9c 2024-09-13T09:09:11.634 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:11.634 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:11.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:11.634 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:11.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:09:11.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:11.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 127.501473984, 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:09:11.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.673191803999998, 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:09:11.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:09:11.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57028 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5128192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:09:11.646 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.380768489999994, 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092372671000000003, 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57028 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:09:11.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251437+0000", 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251439+0000", 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1730000000000002e-06 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251445+0000", 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251450+0000", 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7489999999999998e-06 2024-09-13T09:09:11.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343526+0000", 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092076621999999997 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343605+0000", 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8830000000000002e-05 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343810+0000", 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204544 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.380751966000005, 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:09:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.380672168000004, 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:09:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.380423097000005, 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:09:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:09:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:09:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.380109955999998, 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.379656724, 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:09:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.379632209999997, 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:09:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.379366981000004, 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:09:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:09:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.379079132000001, 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:09:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:09:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:09:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.378617828000003, 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:09:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:09:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.440084478999999, 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:09:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:09:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:09:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.429091159000002, 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:09:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:09:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:09:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.425109261000003, 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:09:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:09:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:09:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.422481267000002, 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:09:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:09:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:09:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:09:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:09:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.420928482999997, 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:09:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:09:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:09:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:09:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.419675167000001, 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:11.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:09:11.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:09:11.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:09:11.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:09:11.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.418573133999999, 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:11.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:09:11.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:09:11.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.417362556999997, 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:11.753 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:09:11.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:09:11.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:11.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:09:11.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:11.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:09:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:09:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:11.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:11.760 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:11.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:11.763 INFO:teuthology.orchestra.run.smithi031.stdout:5392 read 0x783abe3 thru 0x784440a (0x9828 bytes) 2024-09-13T09:09:11.763 INFO:teuthology.orchestra.run.smithi031.stdout:5393 read 0x61d6962 thru 0x61e1fda (0xb679 bytes) 2024-09-13T09:09:11.763 INFO:teuthology.orchestra.run.smithi031.stdout:5394 trunc from 0x8bf4b9c to 0x1404c53 2024-09-13T09:09:11.764 INFO:teuthology.orchestra.run.smithi031.stdout:5395 trunc from 0x1404c53 to 0x765c8c7 2024-09-13T09:09:11.764 INFO:teuthology.orchestra.run.smithi031.stdout:5396 write 0x7814ddf thru 0x781bda4 (0x6fc6 bytes) 2024-09-13T09:09:11.764 INFO:teuthology.orchestra.run.smithi031.stdout:5397 read 0x2d3c702 thru 0x2d3efca (0x28c9 bytes) 2024-09-13T09:09:11.764 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.764 INFO:teuthology.orchestra.run.smithi031.stdout:5401 write 0x98d8ef4 thru 0x98de814 (0x5921 bytes) 2024-09-13T09:09:11.764 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:11.764 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:11.765 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:11.758+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:11.765 INFO:teuthology.orchestra.run.smithi031.stdout:5402 punch from 0x5f439da to 0x5f4aa3b, (0x7061 bytes) 2024-09-13T09:09:11.765 INFO:teuthology.orchestra.run.smithi031.stdout:5403 read 0x2492be9 thru 0x249aca8 (0x80c0 bytes) 2024-09-13T09:09:11.765 INFO:teuthology.orchestra.run.smithi031.stdout:5404 read 0x6c3e010 thru 0x6c4d47a (0xf46b bytes) 2024-09-13T09:09:11.766 INFO:teuthology.orchestra.run.smithi031.stdout:5406 read 0x471d83d thru 0x472113b (0x38ff bytes) 2024-09-13T09:09:11.777 INFO:teuthology.orchestra.run.smithi031.stdout:5407 writesame 0xd057ed0 thru 0xd0657a8 data_size 0x1(0xd8d9 bytes) 2024-09-13T09:09:11.777 INFO:teuthology.orchestra.run.smithi031.stdout:5408 write 0x81063a2 thru 0x8109a63 (0x36c2 bytes) 2024-09-13T09:09:11.777 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:11.778 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:11.785 INFO:teuthology.orchestra.run.smithi031.stdout:5409 read 0xb241fa1 thru 0xb251af7 (0xfb57 bytes) 2024-09-13T09:09:11.785 INFO:teuthology.orchestra.run.smithi031.stdout:5410 read 0xc00ae7d thru 0xc00cc0c (0x1d90 bytes) 2024-09-13T09:09:11.785 INFO:teuthology.orchestra.run.smithi031.stdout:5411 write 0x22a7f78 thru 0x22b08b1 (0x893a bytes) 2024-09-13T09:09:11.792 INFO:teuthology.orchestra.run.smithi031.stdout:5412 punch from 0x9961f46 to 0x996dd71, (0xbe2b bytes) 2024-09-13T09:09:11.792 INFO:teuthology.orchestra.run.smithi031.stdout:5413 writesame 0x7af6afc thru 0x7b06a5e data_size 0x25(0xff63 bytes) 2024-09-13T09:09:11.795 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:11.795 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:11.808 INFO:teuthology.orchestra.run.smithi031.stdout:5416 trunc from 0xd0657a9 to 0x5fdc3a7 2024-09-13T09:09:11.819 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:11.819 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:11.824 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.824 INFO:teuthology.orchestra.run.smithi031.stdout:5418 writesame 0x2f1e060 thru 0x2f20908 data_size 0x1(0x28a9 bytes) 2024-09-13T09:09:11.840 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:11.841 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:11.841 INFO:teuthology.orchestra.run.smithi031.stdout:5419 trunc from 0x5fdc3a7 to 0xd7f38a 2024-09-13T09:09:11.858 INFO:teuthology.orchestra.run.smithi031.stdout:5421 punch from 0xb01ff0 to 0xb07ca7, (0x5cb7 bytes) 2024-09-13T09:09:11.862 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:11.862 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:11.863 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:11.856+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:11.866 INFO:teuthology.orchestra.run.smithi031.stdout:5422 trunc from 0xd7f38a to 0x7932b8e 2024-09-13T09:09:11.874 INFO:teuthology.orchestra.run.smithi031.stdout:5423 read 0x604b8f2 thru 0x604f933 (0x4042 bytes) 2024-09-13T09:09:11.874 INFO:teuthology.orchestra.run.smithi031.stdout:5424 punch from 0x1f9e96e to 0x1fabe4e, (0xd4e0 bytes) 2024-09-13T09:09:11.874 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.874 INFO:teuthology.orchestra.run.smithi031.stdout:5426 write 0x1b58969 thru 0x1b5a9c1 (0x2059 bytes) 2024-09-13T09:09:11.880 INFO:teuthology.orchestra.run.smithi031.stdout:5427 read 0x3a7f4ab thru 0x3a846fa (0x5250 bytes) 2024-09-13T09:09:11.880 INFO:teuthology.orchestra.run.smithi031.stdout:5428 read 0x22a5cbe thru 0x22a7a09 (0x1d4c bytes) 2024-09-13T09:09:11.881 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:11.881 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:11.891 INFO:teuthology.orchestra.run.smithi031.stdout:5429 writesame 0x975ef66 thru 0x97620b7 data_size 0x2(0x3152 bytes) 2024-09-13T09:09:11.891 INFO:teuthology.orchestra.run.smithi031.stdout:5431 punch from 0x52e7c25 to 0x52f6661, (0xea3c bytes) 2024-09-13T09:09:11.892 INFO:teuthology.orchestra.run.smithi031.stdout:5432 read 0x85d0937 thru 0x85dee74 (0xe53e bytes) 2024-09-13T09:09:11.892 INFO:teuthology.orchestra.run.smithi031.stdout:5433 punch from 0xd7b45f to 0xd81610, (0x61b1 bytes) 2024-09-13T09:09:11.892 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.899 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:11.900 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:11.902 INFO:teuthology.orchestra.run.smithi031.stdout:5437 write 0xaf0fef0 thru 0xaf12de8 (0x2ef9 bytes) 2024-09-13T09:09:11.909 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:11.916 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:11.917 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:11.918 INFO:teuthology.orchestra.run.smithi031.stdout:5439 trunc from 0xaf12de9 to 0x53eca1d 2024-09-13T09:09:11.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:11.934 INFO:teuthology.orchestra.run.smithi031.stdout:5441 write 0x31691ca thru 0x3177d67 (0xeb9e bytes) 2024-09-13T09:09:11.939 INFO:teuthology.orchestra.run.smithi031.stdout:5444 read 0x48c635a thru 0x48d1d4c (0xb9f3 bytes) 2024-09-13T09:09:11.939 INFO:teuthology.orchestra.run.smithi031.stdout:5445 writesame 0x25dd8dd thru 0x25e5813 data_size 0x1(0x7f37 bytes) 2024-09-13T09:09:11.964 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:11.962+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:11.995 INFO:teuthology.orchestra.run.smithi031.stdout:5446 writesame 0xb7b4584 thru 0xb7be7f4 data_size 0x1(0xa271 bytes) 2024-09-13T09:09:11.995 INFO:teuthology.orchestra.run.smithi031.stdout:5447 read 0x1f76493 thru 0x1f7d38f (0x6efd bytes) 2024-09-13T09:09:11.995 INFO:teuthology.orchestra.run.smithi031.stdout:5448 read 0x33a996e thru 0x33ae690 (0x4d23 bytes) 2024-09-13T09:09:11.995 INFO:teuthology.orchestra.run.smithi031.stdout:5449 read 0x3ab2b4d thru 0x3ac0dc9 (0xe27d bytes) 2024-09-13T09:09:12.004 INFO:teuthology.orchestra.run.smithi031.stdout:5450 trunc from 0xb7be7f5 to 0x57410e1 2024-09-13T09:09:12.044 INFO:teuthology.orchestra.run.smithi031.stdout:5451 write 0xddacf2b thru 0xddad0bd (0x193 bytes) 2024-09-13T09:09:12.051 INFO:teuthology.orchestra.run.smithi031.stdout:5452 read 0xabcc1ef thru 0xabd1ccb (0x5add bytes) 2024-09-13T09:09:12.051 INFO:teuthology.orchestra.run.smithi031.stdout:5453 read 0xaf8d211 thru 0xaf98d92 (0xbb82 bytes) 2024-09-13T09:09:12.051 INFO:teuthology.orchestra.run.smithi031.stdout:5454 punch from 0xc9f11c0 to 0xc9f36f8, (0x2538 bytes) 2024-09-13T09:09:12.052 INFO:teuthology.orchestra.run.smithi031.stdout:5455 writesame 0xb8764c thru 0xb91c1d data_size 0x32(0xa5d2 bytes) 2024-09-13T09:09:12.064 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.f pg_upmap_items mapping to [1->3,0->5] 2024-09-13T09:09:12.064 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:12.060+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:12.067 INFO:teuthology.orchestra.run.smithi031.stdout:5456 read 0x9a9039a thru 0x9a9a06a (0x9cd1 bytes) 2024-09-13T09:09:12.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:09:12.077 INFO:teuthology.orchestra.run.smithi031.stdout:5458 write 0xe3d4e6c thru 0xe3e4756 (0xf8eb bytes) 2024-09-13T09:09:12.111 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:12.111 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:12.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:12.169 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:12.164+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:12.267 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:12.260+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:12.297 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:12.297 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:12.311 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:12.367 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:12.366+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:12.468 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:12.466+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:12.471 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:12.471 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:12.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:12.569 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:12.564+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:12.605 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:12.605 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:12.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:12.705 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:12.700+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:12.771 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:12.770+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:12.786 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:12.787 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:12.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:12.889 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:12.884+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:12.972 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:12.972 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:12.972 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:12.968+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:12.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:13.077 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:09:13.080 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:13.076+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:13.163 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:13.163 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:13.174 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:13.170+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:13.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:13.275 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:13.274+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:13.347 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:13.347 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:13.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:13.409 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:13.404+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:13.477 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:13.474+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:13.481 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:13.481 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:13.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:13.581 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:13.576+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:13.663 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:13.663 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:13.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:13.721 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:13.716+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:13.781 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:13.778+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:13.840 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:13.841 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:13.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:13.900 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:13.898+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:13.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:13.982 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:13.978+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:13.989 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:14.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:14.083 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:14.076+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:14.086 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:14.086 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:14.102 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:14.102 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:14.118 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:14.118 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:14.134 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:14.155 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:14.157 INFO:teuthology.orchestra.run.smithi031.stdout:5460 writesame 0xa55cbad thru 0xa5691b9 data_size 0x1(0xc60d bytes) 2024-09-13T09:09:14.172 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:14.173 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:14.173 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:14.182 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:14.182 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:14.184 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:14.182+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:14.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:14.199 INFO:teuthology.orchestra.run.smithi031.stdout:5461 read 0x8981a46 thru 0x898c3b2 (0xa96d bytes) 2024-09-13T09:09:14.206 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:14.206 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:14.207 INFO:teuthology.orchestra.run.smithi031.stdout:5462 trunc from 0xe3e4757 to 0x815fbba 2024-09-13T09:09:14.231 INFO:teuthology.orchestra.run.smithi031.stdout:5463 read 0x59d4a50 thru 0x59dff20 (0xb4d1 bytes) 2024-09-13T09:09:14.232 INFO:teuthology.orchestra.run.smithi031.stdout:5465 read 0x7c6af7c thru 0x7c6b0ef (0x174 bytes) 2024-09-13T09:09:14.232 INFO:teuthology.orchestra.run.smithi031.stdout:5468 read 0x6623a51 thru 0x66314a6 (0xda56 bytes) 2024-09-13T09:09:14.232 INFO:teuthology.orchestra.run.smithi031.stdout:5469 punch from 0x3d09074 to 0x3d15485, (0xc411 bytes) 2024-09-13T09:09:14.232 INFO:teuthology.orchestra.run.smithi031.stdout:5470 read 0x4e8aceb thru 0x4e8da96 (0x2dac bytes) 2024-09-13T09:09:14.232 INFO:teuthology.orchestra.run.smithi031.stdout:5472 read 0x64a3936 thru 0x64a4d7c (0x1447 bytes) 2024-09-13T09:09:14.232 INFO:teuthology.orchestra.run.smithi031.stdout:5475 read 0x1340eff thru 0x1342d25 (0x1e27 bytes) 2024-09-13T09:09:14.232 INFO:teuthology.orchestra.run.smithi031.stdout:5476 read 0x44eaf3f thru 0x44ece12 (0x1ed4 bytes) 2024-09-13T09:09:14.233 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:14.233 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:14.243 INFO:teuthology.orchestra.run.smithi031.stdout:5477 write 0x9410bc6 thru 0x9418ce5 (0x8120 bytes) 2024-09-13T09:09:14.252 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:14.252 INFO:teuthology.orchestra.run.smithi031.stdout:5480 read 0x2065409 thru 0x2074887 (0xf47f bytes) 2024-09-13T09:09:14.252 INFO:teuthology.orchestra.run.smithi031.stdout:5481 write 0x48c7832 thru 0x48d6af3 (0xf2c2 bytes) 2024-09-13T09:09:14.262 INFO:teuthology.orchestra.run.smithi031.stdout:5484 writesame 0x3eac37d thru 0x3eb2cd0 data_size 0x6b(0x6954 bytes) 2024-09-13T09:09:14.271 INFO:teuthology.orchestra.run.smithi031.stdout:5485 read 0x16cfb38 thru 0x16d792e (0x7df7 bytes) 2024-09-13T09:09:14.271 INFO:teuthology.orchestra.run.smithi031.stdout:5486 write 0x207524c thru 0x2080963 (0xb718 bytes) 2024-09-13T09:09:14.285 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:14.282+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:14.288 INFO:teuthology.orchestra.run.smithi031.stdout:5487 trunc from 0x9418ce6 to 0xc5c5cf0 2024-09-13T09:09:14.301 INFO:teuthology.orchestra.run.smithi031.stdout:5488 writesame 0x177e88d thru 0x1782402 data_size 0x3b(0x3b76 bytes) 2024-09-13T09:09:14.309 INFO:teuthology.orchestra.run.smithi031.stdout:5489 write 0x73d003c thru 0x73d2c64 (0x2c29 bytes) 2024-09-13T09:09:14.328 INFO:teuthology.orchestra.run.smithi031.stdout:5490 writesame 0xddad44a thru 0xddb3b37 data_size 0x11(0x66ee bytes) 2024-09-13T09:09:14.328 INFO:teuthology.orchestra.run.smithi031.stdout:5491 write 0x208d90e thru 0x209d391 (0xfa84 bytes) 2024-09-13T09:09:14.337 INFO:teuthology.orchestra.run.smithi031.stdout:5492 write 0x8a28551 thru 0x8a2b250 (0x2d00 bytes) 2024-09-13T09:09:14.341 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.227360253000001, 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:09:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:09:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:09:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:09:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.227339486000005, 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:09:14.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:09:14.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.227321634000006, 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:09:14.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:09:14.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:09:14.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.227305068000007, 2024-09-13T09:09:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:09:14.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.227279483000004, 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.363 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.367 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.227260315999999, 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:09:14.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.227243032999993, 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.372 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:09:14.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.226714162999997, 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:09:14.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:09:14.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:14.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.226682257999997, 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:09:14.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.226657309999993, 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:09:14.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.226352745, 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:09:14.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.224590321999997, 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:09:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.224569471999999, 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:09:14.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:09:14.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:14.404 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.224547147999999, 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:09:14.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.224528458999998, 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:09:14.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.224511406999994, 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.224494383000007, 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:09:14.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:09:14.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.214481395999996, 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:09:14.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:09:14.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:09:14.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.834326873000002, 2024-09-13T09:09:14.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.834146451000002, 2024-09-13T09:09:14.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:14.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:09:14.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.441 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:14.442 INFO:teuthology.orchestra.run.smithi031.stdout:5493 read 0x427600d thru 0x42820b7 (0xc0ab bytes) 2024-09-13T09:09:14.442 INFO:teuthology.orchestra.run.smithi031.stdout:5494 trunc from 0xddb3b38 to 0x20845e1 2024-09-13T09:09:14.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:14.444 INFO:teuthology.orchestra.run.smithi031.stdout:5495 write 0xcf18bbe thru 0xcf244ec (0xb92f bytes) 2024-09-13T09:09:14.444 INFO:teuthology.orchestra.run.smithi031.stdout:5496 write 0x90296c6 thru 0x90337bd (0xa0f8 bytes) 2024-09-13T09:09:14.444 INFO:teuthology.orchestra.run.smithi031.stdout:5497 punch from 0x7378660 to 0x7382932, (0xa2d2 bytes) 2024-09-13T09:09:14.444 INFO:teuthology.orchestra.run.smithi031.stdout:5498 punch from 0x784070a to 0x784e0e2, (0xd9d8 bytes) 2024-09-13T09:09:14.444 INFO:teuthology.orchestra.run.smithi031.stdout:5501 write 0xc275319 thru 0xc27a019 (0x4d01 bytes) 2024-09-13T09:09:14.444 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:14.444 INFO:teuthology.orchestra.run.smithi031.stdout:5504 writesame 0xb27021a thru 0xb27738f data_size 0x67(0x7176 bytes) 2024-09-13T09:09:14.444 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:14.438+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:14.445 INFO:teuthology.orchestra.run.smithi031.stdout:5506 punch from 0x7ae45f0 to 0x7ae8ebb, (0x48cb bytes) 2024-09-13T09:09:14.445 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:14.445 INFO:teuthology.orchestra.run.smithi031.stdout:5509 write 0x87eca59 thru 0x87f3ea5 (0x744d bytes) 2024-09-13T09:09:14.454 INFO:teuthology.orchestra.run.smithi031.stdout:5510 read 0xce9629e thru 0xcea1ba9 (0xb90c bytes) 2024-09-13T09:09:14.455 INFO:teuthology.orchestra.run.smithi031.stdout:5511 punch from 0x443e372 to 0x44429a1, (0x462f bytes) 2024-09-13T09:09:14.455 INFO:teuthology.orchestra.run.smithi031.stdout:5512 read 0x9e98828 thru 0x9e9ba64 (0x323d bytes) 2024-09-13T09:09:14.455 INFO:teuthology.orchestra.run.smithi031.stdout:5514 writesame 0x57dc5da thru 0x57e2d1a data_size 0x21(0x6741 bytes) 2024-09-13T09:09:14.463 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:14.464 INFO:teuthology.orchestra.run.smithi031.stdout:5518 read 0x9b5c54e thru 0x9b6c08b (0xfb3e bytes) 2024-09-13T09:09:14.464 INFO:teuthology.orchestra.run.smithi031.stdout:5519 punch from 0x4a735b3 to 0x4a77338, (0x3d85 bytes) 2024-09-13T09:09:14.464 INFO:teuthology.orchestra.run.smithi031.stdout:5520 write 0xc308c9 thru 0xc3956a (0x8ca2 bytes) 2024-09-13T09:09:14.467 INFO:teuthology.orchestra.run.smithi031.stdout:5521 punch from 0x4abb3ae to 0x4abd233, (0x1e85 bytes) 2024-09-13T09:09:14.483 INFO:teuthology.orchestra.run.smithi031.stdout:5522 writesame 0xdaac060 thru 0xdabbc11 data_size 0x3(0xfbb2 bytes) 2024-09-13T09:09:14.483 INFO:teuthology.orchestra.run.smithi031.stdout:5523 read 0x390b697 thru 0x391b5d7 (0xff41 bytes) 2024-09-13T09:09:14.483 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:14.483 INFO:teuthology.orchestra.run.smithi031.stdout:5525 read 0x5c72d22 thru 0x5c7ea6f (0xbd4e bytes) 2024-09-13T09:09:14.483 INFO:teuthology.orchestra.run.smithi031.stdout:5526 punch from 0x2fac0ab to 0x2fbbe3b, (0xfd90 bytes) 2024-09-13T09:09:14.483 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:14.484 INFO:teuthology.orchestra.run.smithi031.stdout:5530 writesame 0xc54a57a thru 0xc55908c data_size 0x1(0xeb13 bytes) 2024-09-13T09:09:14.527 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:14.527 INFO:teuthology.orchestra.run.smithi031.stdout:5532 read 0x68ae8c7 thru 0x68af6fd (0xe37 bytes) 2024-09-13T09:09:14.528 INFO:teuthology.orchestra.run.smithi031.stdout:5534 write 0x6503d19 thru 0x650d8fd (0x9be5 bytes) 2024-09-13T09:09:14.537 INFO:teuthology.orchestra.run.smithi031.stdout:5535 write 0x295de0c thru 0x29650d7 (0x72cc bytes) 2024-09-13T09:09:14.543 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:14.536+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:14.560 INFO:teuthology.orchestra.run.smithi031.stdout:5536 write 0xe669121 thru 0xe66caab (0x398b bytes) 2024-09-13T09:09:14.583 INFO:teuthology.orchestra.run.smithi031.stdout:5537 trunc from 0xe66caac to 0x40bba4b 2024-09-13T09:09:14.587 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:14.587 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:14.587 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:14.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:14.643 INFO:teuthology.orchestra.run.smithi031.stdout:5538 writesame 0xcb2b7ce thru 0xcb386ff data_size 0xb(0xcf32 bytes) 2024-09-13T09:09:14.644 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:14.636+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:14.646 INFO:teuthology.orchestra.run.smithi031.stdout:5541 trunc from 0xcb38700 to 0xe85d5c1 2024-09-13T09:09:14.673 INFO:teuthology.orchestra.run.smithi031.stdout:5544 trunc from 0xe85d5c1 to 0x5b50007 2024-09-13T09:09:14.713 INFO:teuthology.orchestra.run.smithi031.stdout:5545 write 0xec3d798 thru 0xec45aab (0x8314 bytes) 2024-09-13T09:09:14.720 INFO:teuthology.orchestra.run.smithi031.stdout:5546 writesame 0x6a6db47 thru 0x6a6fd5f data_size 0x1d(0x2219 bytes) 2024-09-13T09:09:14.727 INFO:teuthology.orchestra.run.smithi031.stdout:5549 punch from 0x983164d to 0x9840a31, (0xf3e4 bytes) 2024-09-13T09:09:14.728 INFO:teuthology.orchestra.run.smithi031.stdout:5550 read 0x3af88a6 thru 0x3afa5d7 (0x1d32 bytes) 2024-09-13T09:09:14.728 INFO:teuthology.orchestra.run.smithi031.stdout:5551 read 0x7d5add4 thru 0x7d65c1e (0xae4b bytes) 2024-09-13T09:09:14.728 INFO:teuthology.orchestra.run.smithi031.stdout:5552 write 0x819fbf thru 0x828425 (0xe467 bytes) 2024-09-13T09:09:14.732 INFO:teuthology.orchestra.run.smithi031.stdout:5554 write 0xdf30f3f thru 0xdf36b23 (0x5be5 bytes) 2024-09-13T09:09:14.738 INFO:teuthology.orchestra.run.smithi031.stdout:5555 punch from 0x4c38e80 to 0x4c48393, (0xf513 bytes) 2024-09-13T09:09:14.738 INFO:teuthology.orchestra.run.smithi031.stdout:5557 writesame 0x68419da thru 0x6847fe8 data_size 0x3(0x660f bytes) 2024-09-13T09:09:14.745 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:14.742+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:14.773 INFO:teuthology.orchestra.run.smithi031.stdout:5558 trunc from 0xec45aac to 0x511e18c 2024-09-13T09:09:14.783 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:14.783 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:14.799 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:14.806 INFO:teuthology.orchestra.run.smithi031.stdout:5559 read 0x117e431 thru 0x118c0ed (0xdcbd bytes) 2024-09-13T09:09:14.806 INFO:teuthology.orchestra.run.smithi031.stdout:5561 read 0x360d83 thru 0x364e22 (0x40a0 bytes) 2024-09-13T09:09:14.807 INFO:teuthology.orchestra.run.smithi031.stdout:5562 write 0x49141c3 thru 0x49208a4 (0xc6e2 bytes) 2024-09-13T09:09:14.833 INFO:teuthology.orchestra.run.smithi031.stdout:5563 write 0x687a854 thru 0x68809be (0x616b bytes) 2024-09-13T09:09:14.846 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:14.842+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:14.858 INFO:teuthology.orchestra.run.smithi031.stdout:5564 write 0xa729c84 thru 0xa72f3ad (0x572a bytes) 2024-09-13T09:09:14.864 INFO:teuthology.orchestra.run.smithi031.stdout:5565 write 0x464c0ea thru 0x464fb5e (0x3a75 bytes) 2024-09-13T09:09:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:5566 writesame 0xdd3341c thru 0xdd3706b data_size 0x28(0x3c50 bytes) 2024-09-13T09:09:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:5567 write 0x2ca4bc5 thru 0x2caf13b (0xa577 bytes) 2024-09-13T09:09:14.887 INFO:teuthology.orchestra.run.smithi031.stdout:5568 write 0xa33af91 thru 0xa346ae0 (0xbb50 bytes) 2024-09-13T09:09:14.895 INFO:teuthology.orchestra.run.smithi031.stdout:5569 writesame 0x561039f thru 0x5618993 data_size 0x17(0x85f5 bytes) 2024-09-13T09:09:14.903 INFO:teuthology.orchestra.run.smithi031.stdout:5570 write 0xa175fc2 thru 0xa17ba12 (0x5a51 bytes) 2024-09-13T09:09:14.932 INFO:teuthology.orchestra.run.smithi031.stdout:5571 trunc from 0xdd3706c to 0xfc322b 2024-09-13T09:09:14.936 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:14.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 81.982073357999994, 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:09:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:09:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:09:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:09:14.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:09:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:09:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:09:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:09:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:09:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:09:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:14.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:09:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:09:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:09:14.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:09:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:09:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:09:14.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88209 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24748032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:14.946 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699523765999999, 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033308870999999997, 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88209 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235077+0000", 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.22e-06 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235080+0000", 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235083+0000", 2024-09-13T09:09:14.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268290+0000", 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033207532999999997 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268342+0000", 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999997e-05 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268385+0000", 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2874999999999999e-05 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88210 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24772608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699515294999998, 2024-09-13T09:09:14.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033438816000000003, 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88210 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235084+0000", 2024-09-13T09:09:14.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235086+0000", 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235088+0000", 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235091+0000", 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4839999999999998e-06 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268421+0000", 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033329923999999997 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268479+0000", 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8529999999999997e-05 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268523+0000", 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4048999999999998e-05 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88211 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24797184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699506830000004, 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033554247000000002, 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88211 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235094+0000", 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2300000000000001e-06 2024-09-13T09:09:14.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235097+0000", 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.021e-06 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235099+0000", 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268550+0000", 2024-09-13T09:09:14.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033450510000000003 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268593+0000", 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3035999999999999e-05 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268647+0000", 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4336999999999998e-05 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88212 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24821760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699498442000007, 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033675415, 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88212 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235102+0000", 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235105+0000", 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235108+0000", 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.367e-06 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268686+0000", 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033577810999999999 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268737+0000", 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1134999999999997e-05 2024-09-13T09:09:14.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268777+0000", 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0187e-05 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88213 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24846336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699489239000002, 2024-09-13T09:09:14.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033785911000000002, 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88213 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235110+0000", 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235112+0000", 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.728e-06 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235119+0000", 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235124+0000", 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268806+0000", 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033682479000000001 2024-09-13T09:09:14.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268855+0000", 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999998e-05 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268897+0000", 2024-09-13T09:09:14.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1915000000000001e-05 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88214 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24870912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699471603999996, 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033897243000000001, 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88214 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235130+0000", 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235139+0000", 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235151+0000", 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1856e-05 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268933+0000", 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033782874999999997 2024-09-13T09:09:14.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268986+0000", 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2593000000000003e-05 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269026+0000", 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9546999999999999e-05 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88215 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24895488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699446217000002, 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033999110999999999, 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88215 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235153+0000", 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235155+0000", 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8810000000000001e-06 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235159+0000", 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:09:14.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235161+0000", 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269056+0000", 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033895001000000001 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269109+0000", 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269153+0000", 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88216 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24920064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.979 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699436458999998, 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120667, 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88216 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:14.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235173+0000", 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8490000000000001e-06 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235178+0000", 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235180+0000", 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269188+0000", 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034007280000000001 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269245+0000", 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6891000000000001e-05 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269284+0000", 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9561999999999998e-05 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88217 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24944640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699417045000004, 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034230180999999998, 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88217 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:14.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235184+0000", 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3120000000000001e-06 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235187+0000", 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235192+0000", 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269313+0000", 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120708 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269363+0000", 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9453999999999998e-05 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269413+0000", 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0531000000000001e-05 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88218 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24969216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699405487999996, 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034354619000000003, 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88218 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.989 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:14.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235195+0000", 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235199+0000", 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235201+0000", 2024-09-13T09:09:14.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269450+0000", 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034248860999999999 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269502+0000", 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2519000000000003e-05 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269549+0000", 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6740999999999999e-05 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88219 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24993792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699396946999997, 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034466610000000002, 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88219 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:14.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235204+0000", 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.184e-06 2024-09-13T09:09:14.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235207+0000", 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235209+0000", 2024-09-13T09:09:14.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-13T09:09:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269577+0000", 2024-09-13T09:09:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034368528000000002 2024-09-13T09:09:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269628+0000", 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0241999999999998e-05 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269670+0000", 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1875999999999997e-05 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.998 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88220 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25018368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699389096999994, 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034591746999999999, 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88220 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:14.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235212+0000", 2024-09-13T09:09:15.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0839999999999999e-06 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235215+0000", 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9830000000000001e-06 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235217+0000", 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269707+0000", 2024-09-13T09:09:15.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034489989999999998 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269762+0000", 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000003e-05 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269803+0000", 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0181e-05 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.002 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88221 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25042944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699381013999997, 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034709489000000003, 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88221 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235220+0000", 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0640000000000001e-06 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235223+0000", 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-13T09:09:15.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235225+0000", 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269830+0000", 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034604633000000003 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269884+0000", 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067000000000003e-05 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269928+0000", 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4607000000000002e-05 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88222 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25067520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:15.007 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699372867999998, 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034835487999999998, 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88222 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235228+0000", 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235231+0000", 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235233+0000", 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269963+0000", 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034730039999999997 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270016+0000", 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2581999999999999e-05 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270063+0000", 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6718000000000002e-05 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88223 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25092096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699364646999996, 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034953250999999998, 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88223 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235236+0000", 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.127e-06 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235239+0000", 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0180000000000002e-06 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235241+0000", 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270091+0000", 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034849036 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270137+0000", 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6112999999999997e-05 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270189+0000", 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88224 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25116672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699356448000003, 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035071738999999998, 2024-09-13T09:09:15.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88224 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:15.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235244+0000", 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235247+0000", 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7750000000000001e-06 2024-09-13T09:09:15.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235249+0000", 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270226+0000", 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034977019999999998 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270275+0000", 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8791000000000001e-05 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270315+0000", 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9953999999999997e-05 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88225 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25141248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.699348536000002, 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035182109000000003, 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88225 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235252+0000", 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235255+0000", 2024-09-13T09:09:15.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235258+0000", 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270337+0000", 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035079359999999997 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270388+0000", 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0986000000000003e-05 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270433+0000", 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5581000000000003e-05 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93981 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [watch reconnect cookie 140193706156944 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.080500998000005, 2024-09-13T09:09:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116345170000001, 2024-09-13T09:09:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93981 2024-09-13T09:09:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854096+0000", 2024-09-13T09:09:15.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854113+0000", 2024-09-13T09:09:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6759999999999999e-05 2024-09-13T09:09:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854146+0000", 2024-09-13T09:09:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3506000000000001e-05 2024-09-13T09:09:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854169+0000", 2024-09-13T09:09:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2331999999999999e-05 2024-09-13T09:09:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856384+0000", 2024-09-13T09:09:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022150960000000002 2024-09-13T09:09:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856397+0000", 2024-09-13T09:09:15.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3047999999999999e-05 2024-09-13T09:09:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863781+0000", 2024-09-13T09:09:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007384276 2024-09-13T09:09:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863918+0000", 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013628800000000001 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865733+0000", 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018159109999999999 2024-09-13T09:09:15.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93982 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.076837065999996, 2024-09-13T09:09:15.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0094988519999999, 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93982 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857761+0000", 2024-09-13T09:09:15.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857770+0000", 2024-09-13T09:09:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2109999999999998e-06 2024-09-13T09:09:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857789+0000", 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9111999999999999e-05 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857800+0000", 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857854+0000", 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4030999999999998e-05 2024-09-13T09:09:15.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857866+0000", 2024-09-13T09:09:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2275e-05 2024-09-13T09:09:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864650+0000", 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0067835570000001 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864768+0000", 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118019 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:09:15.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865240+0000", 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047203799999999998 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.866250+0000", 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001009935 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867165+0000", 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091516600000000003 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867219+0000", 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4330999999999999e-05 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867262+0000", 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.265e-05 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.048 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:15.053 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:15.054 INFO:teuthology.orchestra.run.smithi031.stdout:5573 punch from 0x17fe2f to 0x18a706, (0xa8d7 bytes) 2024-09-13T09:09:15.054 INFO:teuthology.orchestra.run.smithi031.stdout:5575 punch from 0xc11a4e to 0xc195c3, (0x7b75 bytes) 2024-09-13T09:09:15.054 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:5577 trunc from 0xfc322b to 0x3f72f0f 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:5578 trunc from 0x3f72f0f to 0xd030a42 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:5579 write 0x513832f thru 0x513fc51 (0x7923 bytes) 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:5580 trunc from 0xd030a42 to 0x6d597e4 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:5584 read 0x424a99c thru 0x424f59b (0x4c00 bytes) 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:5585 read 0x2fabed4 thru 0x2fb36f8 (0x7825 bytes) 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:5586 read 0x1b7c01f thru 0x1b7cf14 (0xef6 bytes) 2024-09-13T09:09:15.055 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:15.055 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:15.044+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:15.064 INFO:teuthology.orchestra.run.smithi031.stdout:5588 trunc from 0x6d597e4 to 0x15eeeb0 2024-09-13T09:09:15.094 INFO:teuthology.orchestra.run.smithi031.stdout:5589 read 0x104e8ae thru 0x10570ed (0x8840 bytes) 2024-09-13T09:09:15.094 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:15.094 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:15.095 INFO:teuthology.orchestra.run.smithi031.stdout:5594 read 0xc9e840 thru 0xcabd8b (0xd54c bytes) 2024-09-13T09:09:15.095 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:15.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:09:15.116 INFO:teuthology.orchestra.run.smithi031.stdout:5598 writesame 0x70b80f3 thru 0x70c6ce7 data_size 0x5(0xebf5 bytes) 2024-09-13T09:09:15.116 INFO:teuthology.orchestra.run.smithi031.stdout:5599 write 0x48c37ee thru 0x48cd72e (0x9f41 bytes) 2024-09-13T09:09:15.151 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:15.144+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:15.190 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:15.190 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:15.191 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:15.205 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:15.295 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:15.294+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:15.353 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:15.348+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:15.362 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:15.362 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:15.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:15.463 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:15.462+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:15.542 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:15.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200363653, 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:09:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:15.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:09:15.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20034688, 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:09:15.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:09:15.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20030957500001, 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:09:15.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:15.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200289861, 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:15.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:09:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:09:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200273536, 2024-09-13T09:09:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:09:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:09:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:09:15.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20025767, 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.569 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:09:15.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:09:15.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:09:15.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200241885, 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:09:15.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:15.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:15.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:09:15.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:09:15.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:09:15.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:09:15.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:09:15.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:09:15.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:09:15.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:09:15.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:09:15.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:09:15.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:09:15.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:09:15.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.582 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:15.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200222814, 2024-09-13T09:09:15.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:09:15.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.584 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:09:15.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:15.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:15.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:15.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:09:15.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:09:15.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:09:15.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:09:15.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:09:15.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:09:15.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:09:15.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:09:15.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200206267, 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:09:15.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20017866800001, 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:09:15.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:15.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20016142599999, 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:15.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:09:15.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:09:15.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20013098299999, 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:15.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200112068, 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:09:15.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20009428, 2024-09-13T09:09:15.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:09:15.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:09:15.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200069056, 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:09:15.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:09:15.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:09:15.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.200050852, 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.627 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:09:15.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20003053400001, 2024-09-13T09:09:15.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:15.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.20001198600001, 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.636 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:09:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.19999471200001, 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.641 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:09:15.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:15.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:09:15.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:09:15.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.199977258, 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:15.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.651 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:15.653 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:15.653 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:15.648+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:15.754 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:15.750+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:15.826 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3.e13f538f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.70385627200000001, 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.70387804799999998, 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:15.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:15.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:15.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-09-13T09:09:15.831 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:15.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:15.855 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:15.848+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:15.957 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:15.954+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:16.012 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:16.012 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:16.012 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:16.012 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:16.013 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:16.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:16.072 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:16.070+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:16.086 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:09:16.095 INFO:teuthology.orchestra.run.smithi031.stdout:5601 read 0x6178c68 thru 0x61861fb (0xd594 bytes) 2024-09-13T09:09:16.095 INFO:teuthology.orchestra.run.smithi031.stdout:5602 punch from 0x5901485 to 0x5903a06, (0x2581 bytes) 2024-09-13T09:09:16.095 INFO:teuthology.orchestra.run.smithi031.stdout:5604 punch from 0x139b980 to 0x13a4c4a, (0x92ca bytes) 2024-09-13T09:09:16.117 INFO:teuthology.orchestra.run.smithi031.stdout:5605 writesame 0xa415866 thru 0xa41b95a data_size 0x1(0x60f5 bytes) 2024-09-13T09:09:16.118 INFO:teuthology.orchestra.run.smithi031.stdout:5606 punch from 0x8e2a101 to 0x8e38c2a, (0xeb29 bytes) 2024-09-13T09:09:16.133 INFO:teuthology.orchestra.run.smithi031.stdout:5607 writesame 0xa76344a thru 0xa766ddf data_size 0x6(0x3996 bytes) 2024-09-13T09:09:16.133 INFO:teuthology.orchestra.run.smithi031.stdout:5608 read 0xa388a16 thru 0xa38fbcc (0x71b7 bytes) 2024-09-13T09:09:16.133 INFO:teuthology.orchestra.run.smithi031.stdout:5609 write 0x51fad11 thru 0x5202148 (0x7438 bytes) 2024-09-13T09:09:16.141 INFO:teuthology.orchestra.run.smithi031.stdout:5611 writesame 0xa6313b5 thru 0xa633f68 data_size 0x2(0x2bb4 bytes) 2024-09-13T09:09:16.151 INFO:teuthology.orchestra.run.smithi031.stdout:5612 punch from 0x155a4e3 to 0x155fa54, (0x5571 bytes) 2024-09-13T09:09:16.151 INFO:teuthology.orchestra.run.smithi031.stdout:5614 write 0x359e878 thru 0x35a0063 (0x17ec bytes) 2024-09-13T09:09:16.158 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:16.154+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:16.160 INFO:teuthology.orchestra.run.smithi031.stdout:5615 write 0x8a3d534 thru 0x8a40a28 (0x34f5 bytes) 2024-09-13T09:09:16.170 INFO:teuthology.orchestra.run.smithi031.stdout:5616 trunc from 0xa766de0 to 0x8f2ade1 2024-09-13T09:09:16.190 INFO:teuthology.orchestra.run.smithi031.stdout:5617 write 0x6061553 thru 0x606dce1 (0xc78f bytes) 2024-09-13T09:09:16.216 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:16.216 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:16.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:16.216 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:16.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.216 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.08069812400001, 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:09:16.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:09:16.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:09:16.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.252415944000006, 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:09:16.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:09:16.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.959976105999999, 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:09:16.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.959896307999998, 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.234 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:09:16.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.238 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.959647236999999, 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:09:16.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.959334096000006, 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:09:16.243 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:09:16.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:09:16.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:09:16.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:09:16.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:09:16.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.958880863999994, 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:09:16.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.253 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.958856350000005, 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:09:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.958591120999998, 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:09:16.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:09:16.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.958303271999995, 2024-09-13T09:09:16.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:09:16.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.957841967999997, 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:16.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:09:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:09:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.019308619, 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:16.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:16.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:09:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:09:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:09:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:09:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:16.280 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.008315299000003, 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:09:16.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:09:16.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:09:16.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:16.287 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.004333400999997, 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:09:16.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:09:16.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:09:16.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.001705407000003, 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.295 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:09:16.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:09:16.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.301 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.000152622999998, 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:09:16.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:09:16.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:09:16.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.998899307000002, 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.309 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:09:16.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:09:16.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:09:16.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.997797274, 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:09:16.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:09:16.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:09:16.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.996586696999998, 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:09:16.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:09:16.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3:f1cafc87:::rbd_data.1124651f5f97.000000000000004a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 557056~22319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0888912820000001, 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97276468599999999, 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:09:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092738+0000", 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97003675 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092877+0000", 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139245 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093029+0000", 2024-09-13T09:09:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015224999999999999 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093847+0000", 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081788800000000001 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095250+0000", 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014028529999999999 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095282+0000", 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1961000000000001e-05 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095303+0000", 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:16.336 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:16.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:16.338 INFO:teuthology.orchestra.run.smithi031.stdout:5619 write 0xdc8f9d7 thru 0xdc98ecc (0x94f6 bytes) 2024-09-13T09:09:16.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:5620 read 0xc404120 thru 0xc40aebe (0x6d9f bytes) 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:5622 write 0xcb5e6b1 thru 0xcb60f4a (0x289a bytes) 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:5623 read 0xb098d81 thru 0xb09ae17 (0x2097 bytes) 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:5624 write 0xa243a05 thru 0xa244c28 (0x1224 bytes) 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:5625 write 0xe0f2aab thru 0xe0fcc0d (0xa163 bytes) 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:5628 writesame 0xbfca32 thru 0xc0743e data_size 0x15(0xaa0d bytes) 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:5630 read 0x51a5839 thru 0x51b4a67 (0xf22f bytes) 2024-09-13T09:09:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.342 INFO:teuthology.orchestra.run.smithi031.stdout:5632 write 0xcb192d6 thru 0xcb24b1d (0xb848 bytes) 2024-09-13T09:09:16.342 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.342 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.342 INFO:teuthology.orchestra.run.smithi031.stdout:5636 writesame 0x28867f6 thru 0x288a3d5 data_size 0x10(0x3be0 bytes) 2024-09-13T09:09:16.342 INFO:teuthology.orchestra.run.smithi031.stdout:5637 read 0xa2bf3a7 thru 0xa2c40e5 (0x4d3f bytes) 2024-09-13T09:09:16.342 INFO:teuthology.orchestra.run.smithi031.stdout:5638 punch from 0x315344e to 0x3162c73, (0xf825 bytes) 2024-09-13T09:09:16.342 INFO:teuthology.orchestra.run.smithi031.stdout:5640 writesame 0xa872c65 thru 0xa87305e data_size 0x1(0x3fa bytes) 2024-09-13T09:09:16.342 INFO:teuthology.orchestra.run.smithi031.stdout:5641 write 0xee59799 thru 0xee5ab7c (0x13e4 bytes) 2024-09-13T09:09:16.343 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:16.332+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:16.345 INFO:teuthology.orchestra.run.smithi031.stdout:5642 trunc from 0xee5ab7d to 0x83a6bb6 2024-09-13T09:09:16.380 INFO:teuthology.orchestra.run.smithi031.stdout:5646 trunc from 0x83a6bb6 to 0x31dbdfc 2024-09-13T09:09:16.397 INFO:teuthology.orchestra.run.smithi031.stdout:5647 read 0x245c589 thru 0x245ec68 (0x26e0 bytes) 2024-09-13T09:09:16.417 INFO:teuthology.orchestra.run.smithi031.stdout:5648 write 0x7c12fba thru 0x7c200ee (0xd135 bytes) 2024-09-13T09:09:16.425 INFO:teuthology.orchestra.run.smithi031.stdout:5650 read 0x2a5a3d6 thru 0x2a5c346 (0x1f71 bytes) 2024-09-13T09:09:16.439 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:16.432+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:16.447 INFO:teuthology.orchestra.run.smithi031.stdout:5651 write 0xcfd01f5 thru 0xcfd6d21 (0x6b2d bytes) 2024-09-13T09:09:16.456 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.456 INFO:teuthology.orchestra.run.smithi031.stdout:5654 writesame 0x93f51b7 thru 0x93fb01d data_size 0x1(0x5e67 bytes) 2024-09-13T09:09:16.481 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:16.482 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:16.482 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:16.482 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:16.489 INFO:teuthology.orchestra.run.smithi031.stdout:5655 write 0x73e8aec thru 0x73eec0a (0x611f bytes) 2024-09-13T09:09:16.495 INFO:teuthology.orchestra.run.smithi031.stdout:5656 writesame 0x40be4fe thru 0x40c008c data_size 0x1(0x1b8f bytes) 2024-09-13T09:09:16.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:16.503 INFO:teuthology.orchestra.run.smithi031.stdout:5658 read 0x7709965 thru 0x770ef5f (0x55fb bytes) 2024-09-13T09:09:16.504 INFO:teuthology.orchestra.run.smithi031.stdout:5659 read 0xe73666 thru 0xe82e64 (0xf7ff bytes) 2024-09-13T09:09:16.504 INFO:teuthology.orchestra.run.smithi031.stdout:5660 punch from 0x10244e9 to 0x102a2fe, (0x5e15 bytes) 2024-09-13T09:09:16.505 INFO:teuthology.orchestra.run.smithi031.stdout:5661 writesame 0x67b833 thru 0x6886f0 data_size 0x3(0xcebe bytes) 2024-09-13T09:09:16.517 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:16.517 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:16.519 INFO:teuthology.orchestra.run.smithi031.stdout:5662 read 0x37b56cf thru 0x37bd240 (0x7b72 bytes) 2024-09-13T09:09:16.520 INFO:teuthology.orchestra.run.smithi031.stdout:5664 writesame 0x7a26fe1 thru 0x7a30cf6 data_size 0x1(0x9d16 bytes) 2024-09-13T09:09:16.536 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:16.536 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:16.540 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:16.538+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:16.545 INFO:teuthology.orchestra.run.smithi031.stdout:5665 write 0xa513778 thru 0xa516b2f (0x33b8 bytes) 2024-09-13T09:09:16.551 INFO:teuthology.orchestra.run.smithi031.stdout:5666 punch from 0x42d364a to 0x42d559e, (0x1f54 bytes) 2024-09-13T09:09:16.551 INFO:teuthology.orchestra.run.smithi031.stdout:5668 read 0x9753dcd thru 0x97597d0 (0x5a04 bytes) 2024-09-13T09:09:16.551 INFO:teuthology.orchestra.run.smithi031.stdout:5669 write 0xc6d54e8 thru 0xc6dde09 (0x8922 bytes) 2024-09-13T09:09:16.553 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:16.553 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:16.557 INFO:teuthology.orchestra.run.smithi031.stdout:5671 write 0xbc56169 thru 0xbc5a153 (0x3feb bytes) 2024-09-13T09:09:16.571 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:16.571 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:16.573 INFO:teuthology.orchestra.run.smithi031.stdout:5672 write 0xeca358f thru 0xecad2c8 (0x9d3a bytes) 2024-09-13T09:09:16.578 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.582 INFO:teuthology.orchestra.run.smithi031.stdout:5676 trunc from 0xecad2c9 to 0xc11340d 2024-09-13T09:09:16.590 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:16.600 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.608 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:16.609 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:16.617 INFO:teuthology.orchestra.run.smithi031.stdout:5680 trunc from 0xc11340d to 0x16f22de 2024-09-13T09:09:16.630 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:16.630 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:16.636 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.641 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:16.638+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:16.651 INFO:teuthology.orchestra.run.smithi031.stdout:5682 writesame 0x40aac08 thru 0x40b233e data_size 0x3(0x7737 bytes) 2024-09-13T09:09:16.651 INFO:teuthology.orchestra.run.smithi031.stdout:5683 writesame 0x3dade2d thru 0x3dbd441 data_size 0x53(0xf615 bytes) 2024-09-13T09:09:16.653 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:16.653 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:16.672 INFO:teuthology.orchestra.run.smithi031.stdout:5684 write 0x586982d thru 0x586fc1d (0x63f1 bytes) 2024-09-13T09:09:16.675 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:16.676 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:16.677 INFO:teuthology.orchestra.run.smithi031.stdout:5685 read 0x34fa438 thru 0x3506878 (0xc441 bytes) 2024-09-13T09:09:16.677 INFO:teuthology.orchestra.run.smithi031.stdout:5687 read 0x4b77b67 thru 0x4b77d49 (0x1e3 bytes) 2024-09-13T09:09:16.677 INFO:teuthology.orchestra.run.smithi031.stdout:5688 read 0x17c6b07 thru 0x17d5872 (0xed6c bytes) 2024-09-13T09:09:16.677 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.684 INFO:teuthology.orchestra.run.smithi031.stdout:5691 trunc from 0x586fc1e to 0x983da8 2024-09-13T09:09:16.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:16.739 INFO:teuthology.orchestra.run.smithi031.stdout:5694 write 0xc743ebd thru 0xc7442e3 (0x427 bytes) 2024-09-13T09:09:16.741 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:16.738+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:16.743 INFO:teuthology.orchestra.run.smithi031.stdout:5695 writesame 0xc20f00e thru 0xc20f24e data_size 0x1(0x241 bytes) 2024-09-13T09:09:16.772 INFO:teuthology.orchestra.run.smithi031.stdout:5696 trunc from 0xc7442e4 to 0x20f6acc 2024-09-13T09:09:16.792 INFO:teuthology.orchestra.run.smithi031.stdout:5697 read 0x1a36221 thru 0x1a3a05c (0x3e3c bytes) 2024-09-13T09:09:16.816 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:16.816 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:16.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:16.837 INFO:teuthology.orchestra.run.smithi031.stdout:5700 write 0xe859d48 thru 0xe85b49a (0x1753 bytes) 2024-09-13T09:09:16.842 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:16.838+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:16.845 INFO:teuthology.orchestra.run.smithi031.stdout:5703 writesame 0x95271c1 thru 0x952880c data_size 0x2(0x164c bytes) 2024-09-13T09:09:16.855 INFO:teuthology.orchestra.run.smithi031.stdout:5704 write 0xbadbd95 thru 0xbadce21 (0x108d bytes) 2024-09-13T09:09:16.862 INFO:teuthology.orchestra.run.smithi031.stdout:5705 read 0x23bc1aa thru 0x23c6500 (0xa357 bytes) 2024-09-13T09:09:16.862 INFO:teuthology.orchestra.run.smithi031.stdout:5706 read 0xaa90c18 thru 0xaa9a2b2 (0x969b bytes) 2024-09-13T09:09:16.862 INFO:teuthology.orchestra.run.smithi031.stdout:5707 read 0xbf54ed2 thru 0xbf60179 (0xb2a8 bytes) 2024-09-13T09:09:16.862 INFO:teuthology.orchestra.run.smithi031.stdout:5708 write 0x9065c5a thru 0x906fce8 (0xa08f bytes) 2024-09-13T09:09:16.892 INFO:teuthology.orchestra.run.smithi031.stdout:5709 trunc from 0xe85b49b to 0x3c478ef 2024-09-13T09:09:16.934 INFO:teuthology.orchestra.run.smithi031.stdout:5710 write 0x619c4a1 thru 0x619d9e3 (0x1543 bytes) 2024-09-13T09:09:16.941 INFO:teuthology.orchestra.run.smithi031.stdout:5711 write 0x1ebe674 thru 0x1ec66c2 (0x804f bytes) 2024-09-13T09:09:16.943 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:16.942+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:16.948 INFO:teuthology.orchestra.run.smithi031.stdout:5712 writesame 0x2ae933e thru 0x2af7287 data_size 0x15(0xdf4a bytes) 2024-09-13T09:09:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:5714 read 0x48d0623 thru 0x48d8b3a (0x8518 bytes) 2024-09-13T09:09:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:5715 write 0x5779c6b thru 0x5779df2 (0x188 bytes) 2024-09-13T09:09:16.966 INFO:teuthology.orchestra.run.smithi031.stdout:5716 punch from 0xd09b3e to 0xd0bc2e, (0x20f0 bytes) 2024-09-13T09:09:16.968 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:16.968 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:16.981 INFO:teuthology.orchestra.run.smithi031.stdout:5717 write 0x76986b7 thru 0x7698b22 (0x46c bytes) 2024-09-13T09:09:16.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:16.989 INFO:teuthology.orchestra.run.smithi031.stdout:5719 write 0x318770f thru 0x3192d20 (0xb612 bytes) 2024-09-13T09:09:16.996 INFO:teuthology.orchestra.run.smithi031.stdout:5720 write 0x2d78d97 thru 0x2d81801 (0x8a6b bytes) 2024-09-13T09:09:17.000 INFO:teuthology.orchestra.run.smithi031.stdout:5721 punch from 0x5e378e8 to 0x5e44e58, (0xd570 bytes) 2024-09-13T09:09:17.000 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:17.001 INFO:teuthology.orchestra.run.smithi031.stdout:5724 read 0x480c1fb thru 0x480e94d (0x2753 bytes) 2024-09-13T09:09:17.001 INFO:teuthology.orchestra.run.smithi031.stdout:5726 clone 44 order 18 su 8192 sc 14 2024-09-13T09:09:17.044 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:17.036+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:17.089 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1] out_osds: [7, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:17.089 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:09:17.089 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T09:09:17.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:09:17.113 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:17.113 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:17.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:17.173 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:17.168+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:17.233 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone43 2024-09-13T09:09:17.245 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:17.242+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:17.313 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:17.313 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:17.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:17.373 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:17.368+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:17.421 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 120, 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T09:06:44.149006+0000", 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T09:09:17.083337+0000", 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T09:06:55.611487+0000", 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T09:08:59.952135+0000", 2024-09-13T09:09:17.432 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:09:17.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:06:55.131330+0000", 2024-09-13T09:09:17.434 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T09:09:17.435 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T09:09:17.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:17.437 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:17.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T09:09:17.439 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:17.440 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:02.743102+0000", 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:17.441 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:17.442 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-09-13T09:09:17.443 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:17.444 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:17.445 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:17.446 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9699997901916504, 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9699997901916504, 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.62999999523162842, 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 3.130000114440918, 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 3.130000114440918, 2024-09-13T09:09:17.447 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:12.517158+0000", 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:17.448 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T09:09:17.449 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "120", 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:17.450 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 47, 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 120, 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:17.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:17.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:17.453 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.75, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.5, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.5, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:17.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "33c6fc7d-9828-4500-b103-b96d1cd7f420", 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:17.455 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 107, 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T09:09:17.459 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6818/3291756747", 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6822/3291756747", 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6830/3291756747", 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6826/3291756747", 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:17.460 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "869ea3c5-8e6e-4b49-b70e-083f242a7aae", 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.999481201171875, 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:17.461 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 117, 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.463 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/533520566", 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/533520566", 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/533520566", 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/533520566", 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e144d496-76ee-46e0-bdc6-135c77128535", 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:09:17.467 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:17.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:17.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:17.470 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:17.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6819/2253604482", 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6823/2253604482", 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/2253604482", 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6827/2253604482", 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.472 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "696b9414-9c19-4984-a8ff-14548e46b357", 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 118, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:17.473 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:17.475 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.476 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.477 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/3367214693", 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/3367214693", 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/3367214693", 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/3367214693", 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ac58040d-9b54-42ca-b143-1816abb54816", 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:17.479 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6800", 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6801", 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.480 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6802", 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6803", 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:17.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6806", 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6807", 2024-09-13T09:09:17.482 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6804", 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6805", 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6801/3778074750", 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6803/3778074750", 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6807/3778074750", 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6805/3778074750", 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:17.484 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2", 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:17.485 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6808", 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.486 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6809", 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6811", 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6813", 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6819", 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6824", 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6815", 2024-09-13T09:09:17.489 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6817", 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6809/1926603087", 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6813/1926603087", 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6824/1926603087", 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6817/1926603087", 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:17.490 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d", 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:09:17.491 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6820", 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6821", 2024-09-13T09:09:17.492 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6823", 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.493 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6825", 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6830", 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.494 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6831", 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6827", 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6828", 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6821/766305500", 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6825/766305500", 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6831/766305500", 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6828/766305500", 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:17.496 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f36c332c-9497-47e8-b943-f6b6dcaebeb8", 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:09:17.497 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6810", 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6812", 2024-09-13T09:09:17.498 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6814", 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:17.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6816", 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6826", 2024-09-13T09:09:17.501 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6829", 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:17.502 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6818", 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6822", 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6812/1032956039", 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6816/1032956039", 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6829/1032956039", 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6822/1032956039", 2024-09-13T09:09:17.503 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:09:17.504 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 16238, 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.183436+0000", 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:17.505 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.264080+0000", 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:17.506 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.253587+0000", 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.244429+0000", 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.507 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.369458+0000", 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:17.508 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.158089+0000", 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:17.509 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.217295+0000", 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.332254+0000", 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:17.510 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.a", 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: 4, 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: 2 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [ 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.f", 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:17.511 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 1, 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 3 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T09:09:17.512 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T09:09:17.513 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T09:09:17.514 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T09:09:17.514 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T09:09:17.514 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:09:17.514 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:09:17.514 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T09:09:17.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:17.514 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:17.516 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:17.516 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:17.517 INFO:tasks.thrashosds.thrasher:j is {'epoch': 120, 'fsid': 'fe16bf6d-4166-4417-80fd-1123617a33d3', 'created': '2024-09-13T09:06:44.149006+0000', 'modified': '2024-09-13T09:09:17.083337+0000', 'last_up_change': '2024-09-13T09:06:55.611487+0000', 'last_in_change': '2024-09-13T09:08:59.952135+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:06:55.131330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:07:02.743102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.96999979019165, 'score_stable': 4.96999979019165, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 3.130000114440918, 'raw_score_stable': 3.130000114440918, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:07:12.517158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 120, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '33c6fc7d-9828-4500-b103-b96d1cd7f420', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6818', 'nonce': 3291756747}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6822', 'nonce': 3291756747}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6830', 'nonce': 3291756747}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6826', 'nonce': 3291756747}]}, 'public_addr': '172.21.15.31:6818/3291756747', 'cluster_addr': '172.21.15.31:6822/3291756747', 'heartbeat_back_addr': '172.21.15.31:6830/3291756747', 'heartbeat_front_addr': '172.21.15.31:6826/3291756747', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.999481201171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 533520566}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 533520566}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 533520566}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 533520566}]}, 'public_addr': '172.21.15.31:6801/533520566', 'cluster_addr': '172.21.15.31:6803/533520566', 'heartbeat_back_addr': '172.21.15.31:6807/533520566', 'heartbeat_front_addr': '172.21.15.31:6805/533520566', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e144d496-76ee-46e0-bdc6-135c77128535', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6817', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2253604482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6821', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2253604482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6829', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 2253604482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6825', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 2253604482}]}, 'public_addr': '172.21.15.31:6819/2253604482', 'cluster_addr': '172.21.15.31:6823/2253604482', 'heartbeat_back_addr': '172.21.15.31:6831/2253604482', 'heartbeat_front_addr': '172.21.15.31:6827/2253604482', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '696b9414-9c19-4984-a8ff-14548e46b357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 3367214693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 3367214693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 3367214693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 3367214693}]}, 'public_addr': '172.21.15.31:6809/3367214693', 'cluster_addr': '172.21.15.31:6811/3367214693', 'heartbeat_back_addr': '172.21.15.31:6815/3367214693', 'heartbeat_front_addr': '172.21.15.31:6813/3367214693', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ac58040d-9b54-42ca-b143-1816abb54816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6800', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6801', 'nonce': 3778074750}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 3778074750}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 3778074750}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 3778074750}]}, 'public_addr': '172.21.15.60:6801/3778074750', 'cluster_addr': '172.21.15.60:6803/3778074750', 'heartbeat_back_addr': '172.21.15.60:6807/3778074750', 'heartbeat_front_addr': '172.21.15.60:6805/3778074750', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1926603087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6811', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 1926603087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6819', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6824', 'nonce': 1926603087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6815', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 1926603087}]}, 'public_addr': '172.21.15.60:6809/1926603087', 'cluster_addr': '172.21.15.60:6813/1926603087', 'heartbeat_back_addr': '172.21.15.60:6824/1926603087', 'heartbeat_front_addr': '172.21.15.60:6817/1926603087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 766305500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6823', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 766305500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 766305500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6827', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6828', 'nonce': 766305500}]}, 'public_addr': '172.21.15.60:6821/766305500', 'cluster_addr': '172.21.15.60:6825/766305500', 'heartbeat_back_addr': '172.21.15.60:6831/766305500', 'heartbeat_front_addr': '172.21.15.60:6828/766305500', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f36c332c-9497-47e8-b943-f6b6dcaebeb8', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6812', 'nonce': 1032956039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6816', 'nonce': 1032956039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 1032956039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6822', 'nonce': 1032956039}]}, 'public_addr': '172.21.15.60:6812/1032956039', 'cluster_addr': '172.21.15.60:6816/1032956039', 'heartbeat_back_addr': '172.21.15.60:6829/1032956039', 'heartbeat_front_addr': '172.21.15.60:6822/1032956039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 16238, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.183436+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.264080+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.253587+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.244429+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.369458+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.158089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.217295+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.332254+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.a', 'osds': [4, 2]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 1, 'to': 3}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:09:17.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:09:17.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:17.519 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:17.508+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:17.519 INFO:teuthology.orchestra.run.smithi031.stdout:5727 punch from 0x465e737 to 0x465f174, (0xa3d bytes) 2024-09-13T09:09:17.519 INFO:teuthology.orchestra.run.smithi031.stdout:5728 read 0x2d3d8b1 thru 0x2d45b82 (0x82d2 bytes) 2024-09-13T09:09:17.519 INFO:teuthology.orchestra.run.smithi031.stdout:5729 write 0x762e805 thru 0x7631162 (0x295e bytes) 2024-09-13T09:09:17.519 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:17.519 INFO:teuthology.orchestra.run.smithi031.stdout:5732 writesame 0x76f4ddc thru 0x7701eec data_size 0x17(0xd111 bytes) 2024-09-13T09:09:17.520 INFO:teuthology.orchestra.run.smithi031.stdout:5733 punch from 0x32c828f to 0x32d578e, (0xd4ff bytes) 2024-09-13T09:09:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:5735 writesame 0xaded299 thru 0xadf069d data_size 0x17(0x3405 bytes) 2024-09-13T09:09:17.536 INFO:teuthology.orchestra.run.smithi031.stdout:5736 write 0x8da54ef thru 0x8db2b65 (0xd677 bytes) 2024-09-13T09:09:17.551 INFO:teuthology.orchestra.run.smithi031.stdout:5737 write 0x7246335 thru 0x724ed0a (0x89d6 bytes) 2024-09-13T09:09:17.562 INFO:teuthology.orchestra.run.smithi031.stdout:5738 read 0x897de3a thru 0x8980207 (0x23ce bytes) 2024-09-13T09:09:17.562 INFO:teuthology.orchestra.run.smithi031.stdout:5739 punch from 0x7c1f712 to 0x7c25c15, (0x6503 bytes) 2024-09-13T09:09:17.562 INFO:teuthology.orchestra.run.smithi031.stdout:5740 write 0x9e6dc52 thru 0x9e70be8 (0x2f97 bytes) 2024-09-13T09:09:17.569 INFO:teuthology.orchestra.run.smithi031.stdout:5741 punch from 0x7167b8b to 0x716d32f, (0x57a4 bytes) 2024-09-13T09:09:17.569 INFO:teuthology.orchestra.run.smithi031.stdout:5743 writesame 0x586fd89 thru 0x5875e11 data_size 0x1(0x6089 bytes) 2024-09-13T09:09:17.590 INFO:teuthology.orchestra.run.smithi031.stdout:5746 write 0x76c4ca4 thru 0x76ce0ed (0x944a bytes) 2024-09-13T09:09:17.603 INFO:teuthology.orchestra.run.smithi031.stdout:5750 trunc from 0xadf069e to 0xb3025e4 2024-09-13T09:09:17.610 INFO:teuthology.orchestra.run.smithi031.stdout:5752 write 0x836cabd thru 0x837c8fd (0xfe41 bytes) 2024-09-13T09:09:17.616 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:17.612+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:17.624 INFO:teuthology.orchestra.run.smithi031.stdout:5753 writesame 0x2802e12 thru 0x280ab43 data_size 0x5(0x7d32 bytes) 2024-09-13T09:09:17.637 INFO:teuthology.orchestra.run.smithi031.stdout:5754 read 0x28f1cb0 thru 0x28f55c7 (0x3918 bytes) 2024-09-13T09:09:17.637 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:17.637 INFO:teuthology.orchestra.run.smithi031.stdout:5756 read 0x111fd2f thru 0x112daef (0xddc1 bytes) 2024-09-13T09:09:17.637 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:17.637 INFO:teuthology.orchestra.run.smithi031.stdout:5760 write 0x60da3a4 thru 0x60dd7a2 (0x33ff bytes) 2024-09-13T09:09:17.654 INFO:teuthology.orchestra.run.smithi031.stdout:5761 writesame 0xc8b657c thru 0xc8c6340 data_size 0x47(0xfdc5 bytes) 2024-09-13T09:09:17.654 INFO:teuthology.orchestra.run.smithi031.stdout:5762 write 0x350553e thru 0x35058e4 (0x3a7 bytes) 2024-09-13T09:09:17.667 INFO:teuthology.orchestra.run.smithi031.stdout:5763 write 0xe0b76db thru 0xe0c51a1 (0xdac7 bytes) 2024-09-13T09:09:17.677 INFO:teuthology.orchestra.run.smithi031.stdout:5765 write 0x816744 thru 0x81bf48 (0x5805 bytes) 2024-09-13T09:09:17.689 INFO:teuthology.orchestra.run.smithi031.stdout:5766 write 0x325544d thru 0x3258cd7 (0x388b bytes) 2024-09-13T09:09:17.697 INFO:teuthology.orchestra.run.smithi031.stdout:5767 punch from 0x7c5b58d to 0x7c66654, (0xb0c7 bytes) 2024-09-13T09:09:17.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:17.698 INFO:teuthology.orchestra.run.smithi031.stdout:5769 read 0x3f4202d thru 0x3f4898e (0x6962 bytes) 2024-09-13T09:09:17.698 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:17.698 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:17.709 INFO:teuthology.orchestra.run.smithi031.stdout:5773 trunc from 0xe0c51a2 to 0x5740a49 2024-09-13T09:09:17.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:17.717 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:17.714+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:17.759 INFO:teuthology.orchestra.run.smithi031.stdout:5774 read 0x39949a8 thru 0x399c170 (0x77c9 bytes) 2024-09-13T09:09:17.769 INFO:teuthology.orchestra.run.smithi031.stdout:5775 write 0x7144336 thru 0x7152cc7 (0xe992 bytes) 2024-09-13T09:09:17.781 INFO:teuthology.orchestra.run.smithi031.stdout:5777 writesame 0x5e6462b thru 0x5e64a40 data_size 0x1(0x416 bytes) 2024-09-13T09:09:17.801 INFO:teuthology.orchestra.run.smithi031.stdout:5780 write 0xde5ebd5 thru 0xde6872d (0x9b59 bytes) 2024-09-13T09:09:17.812 INFO:teuthology.orchestra.run.smithi031.stdout:5781 writesame 0x84f1cb2 thru 0x84f8b2d data_size 0x6(0x6e7c bytes) 2024-09-13T09:09:17.818 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:17.814+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:17.821 INFO:teuthology.orchestra.run.smithi031.stdout:5782 writesame 0xce159de thru 0xce21ee9 data_size 0x2(0xc50c bytes) 2024-09-13T09:09:17.851 INFO:teuthology.orchestra.run.smithi031.stdout:5783 write 0xe38f1ab thru 0xe39a898 (0xb6ee bytes) 2024-09-13T09:09:17.862 INFO:teuthology.orchestra.run.smithi031.stdout:5784 punch from 0xb0dd105 to 0xb0ea84c, (0xd747 bytes) 2024-09-13T09:09:17.862 INFO:teuthology.orchestra.run.smithi031.stdout:5785 punch from 0x932b925 to 0x933331d, (0x79f8 bytes) 2024-09-13T09:09:17.877 INFO:teuthology.orchestra.run.smithi031.stdout:5787 trunc from 0xe39a899 to 0x301b666 2024-09-13T09:09:17.894 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:17.894 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:17.911 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:09:17.912 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:09:17.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:17.942 INFO:teuthology.orchestra.run.smithi031.stdout:5788 punch from 0x2ba56b4 to 0x2ba8a94, (0x33e0 bytes) 2024-09-13T09:09:17.944 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-13T09:09:17.094860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1531605541,"num_objects":1576,"num_object_clones":341,"num_object_copies":3152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1576,"num_whiteouts":580,"num_read":183829,"num_read_kb":613988,"num_write":20595,"num_write_kb":315848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1657,"num_bytes_recovered":168133482,"num_keys_recovered":370,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16256,"ondisk_log_size":16256,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":636936,"kb_used_data":347632,"kb_used_omap":9,"kb_used_meta":289142,"kb_avail":565594104,"statfs":{"total":579820584960,"available":579168362496,"internally_reserved":0,"allocated":355975168,"data_stored":383632624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":296082087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12061322,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":1,"num_read":3905,"num_read_kb":4737,"num_write":1119,"num_write_kb":6513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":2807142,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.011870"},"pg_stats":[{"pgid":"3.1f","version":"114'462","reported_seq":4944,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T09:09:12.059005+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:09:12.059005+0000","last_peered":"2024-09-13T09:09:12.059005+0000","last_clean":"2024-09-13T09:09:12.059005+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:09:12.059005+0000","last_undegraded":"2024-09-13T09:09:12.059005+0000","last_fullsized":"2024-09-13T09:09:12.059005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":37991697,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":5508,"num_read_kb":16320,"num_write":599,"num_write_kb":5088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"115'886","reported_seq":3364,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T09:09:13.069207+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:09:13.069207+0000","last_peered":"2024-09-13T09:09:13.069207+0000","last_clean":"2024-09-13T09:09:13.069207+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:09:13.069207+0000","last_undegraded":"2024-09-13T09:09:13.069207+0000","last_fullsized":"2024-09-13T09:09:13.069207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":46552965,"num_objects":56,"num_object_clones":10,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":20,"num_read":6471,"num_read_kb":14129,"num_write":1212,"num_write_kb":5622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"119'1312","reported_seq":4995,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T09:09:16.085988+0000","last_change":"2024-09-13T09:08:38.210944+0000","last_active":"2024-09-13T09:09:16.085988+0000","last_peered":"2024-09-13T09:09:16.085988+0000","last_clean":"2024-09-13T09:09:16.085988+0000","last_became_active":"2024-09-13T09:08:38.208191+0000","last_became_peered":"2024-09-13T09:08:38.208191+0000","last_unstale":"2024-09-13T09:09:16.085988+0000","last_undegraded":"2024-09-13T09:09:16.085988+0000","last_fullsized":"2024-09-13T09:09:16.085988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1312,"log_dups_size":0,"ondisk_log_size":1312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:04.686300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":54604582,"num_objects":61,"num_object_clones":16,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":28,"num_read":8646,"num_read_kb":33476,"num_write":2175,"num_write_kb":16994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"117'446","reported_seq":3078,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.078323+0000","last_change":"2024-09-13T09:08:38.212078+0000","last_active":"2024-09-13T09:09:15.078323+0000","last_peered":"2024-09-13T09:09:15.078323+0000","last_clean":"2024-09-13T09:09:15.078323+0000","last_became_active":"2024-09-13T09:08:38.209451+0000","last_became_peered":"2024-09-13T09:08:38.209451+0000","last_unstale":"2024-09-13T09:09:15.078323+0000","last_undegraded":"2024-09-13T09:09:15.078323+0000","last_fullsized":"2024-09-13T09:09:15.078323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:59.672566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":77523185,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":16,"num_read":4456,"num_read_kb":33259,"num_write":596,"num_write_kb":15254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"117'532","reported_seq":21173,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T09:09:14.730920+0000","last_change":"2024-09-13T09:08:38.212581+0000","last_active":"2024-09-13T09:09:14.730920+0000","last_peered":"2024-09-13T09:09:14.730920+0000","last_clean":"2024-09-13T09:09:14.730920+0000","last_became_active":"2024-09-13T09:08:38.210333+0000","last_became_peered":"2024-09-13T09:08:38.210333+0000","last_unstale":"2024-09-13T09:09:14.730920+0000","last_undegraded":"2024-09-13T09:09:14.730920+0000","last_fullsized":"2024-09-13T09:09:14.730920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:53.493070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":93670173,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":17,"num_read":22603,"num_read_kb":45435,"num_write":617,"num_write_kb":18412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"117'544","reported_seq":5040,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T09:09:16.189581+0000","last_change":"2024-09-13T09:08:09.864519+0000","last_active":"2024-09-13T09:09:16.189581+0000","last_peered":"2024-09-13T09:09:16.189581+0000","last_clean":"2024-09-13T09:09:16.189581+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:09:16.189581+0000","last_undegraded":"2024-09-13T09:09:16.189581+0000","last_fullsized":"2024-09-13T09:09:16.189581+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":60629886,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":23,"num_read":6968,"num_read_kb":22029,"num_write":716,"num_write_kb":18966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"117'299","reported_seq":1338,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T09:09:14.940639+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:09:14.940639+0000","last_peered":"2024-09-13T09:09:14.940639+0000","last_clean":"2024-09-13T09:09:14.940639+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:09:14.940639+0000","last_undegraded":"2024-09-13T09:09:14.940639+0000","last_fullsized":"2024-09-13T09:09:14.940639+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":14900211,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":1309,"num_read_kb":6337,"num_write":284,"num_write_kb":6926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"114'547","reported_seq":2779,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T09:09:12.058763+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:09:12.058763+0000","last_peered":"2024-09-13T09:09:12.058763+0000","last_clean":"2024-09-13T09:09:12.058763+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:09:12.058763+0000","last_undegraded":"2024-09-13T09:09:12.058763+0000","last_fullsized":"2024-09-13T09:09:12.058763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":45848292,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":19,"num_read":4007,"num_read_kb":13116,"num_write":651,"num_write_kb":9842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"114'416","reported_seq":4576,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T09:09:12.058978+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:09:12.058978+0000","last_peered":"2024-09-13T09:09:12.058978+0000","last_clean":"2024-09-13T09:09:12.058978+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:09:12.058978+0000","last_undegraded":"2024-09-13T09:09:12.058978+0000","last_fullsized":"2024-09-13T09:09:12.058978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":54939023,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":5135,"num_read_kb":28859,"num_write":435,"num_write_kb":17610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"117'616","reported_seq":1549,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.079382+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:09:15.079382+0000","last_peered":"2024-09-13T09:09:15.079382+0000","last_clean":"2024-09-13T09:09:15.079382+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:09:15.079382+0000","last_undegraded":"2024-09-13T09:09:15.079382+0000","last_fullsized":"2024-09-13T09:09:15.079382+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":35472392,"num_objects":45,"num_object_clones":13,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":2677,"num_read_kb":12757,"num_write":897,"num_write_kb":9405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"117'529","reported_seq":3503,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T09:09:14.536283+0000","last_change":"2024-09-13T09:08:38.222834+0000","last_active":"2024-09-13T09:09:14.536283+0000","last_peered":"2024-09-13T09:09:14.536283+0000","last_clean":"2024-09-13T09:09:14.536283+0000","last_became_active":"2024-09-13T09:08:38.221484+0000","last_became_peered":"2024-09-13T09:08:38.221484+0000","last_unstale":"2024-09-13T09:09:14.536283+0000","last_undegraded":"2024-09-13T09:09:14.536283+0000","last_fullsized":"2024-09-13T09:09:14.536283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:04.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":92172166,"num_objects":54,"num_object_clones":16,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":16,"num_read":4405,"num_read_kb":22942,"num_write":551,"num_write_kb":14626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"114'561","reported_seq":18504,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.081846+0000","last_change":"2024-09-13T09:07:15.879981+0000","last_active":"2024-09-13T09:09:15.081846+0000","last_peered":"2024-09-13T09:09:15.081846+0000","last_clean":"2024-09-13T09:09:15.081846+0000","last_became_active":"2024-09-13T09:07:13.859089+0000","last_became_peered":"2024-09-13T09:07:13.859089+0000","last_unstale":"2024-09-13T09:09:15.081846+0000","last_undegraded":"2024-09-13T09:09:15.081846+0000","last_fullsized":"2024-09-13T09:09:15.081846+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:30:47.730669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":46468143,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":16,"num_read":20305,"num_read_kb":32181,"num_write":523,"num_write_kb":9579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":123,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T09:09:16.086560+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:09:16.086560+0000","last_peered":"2024-09-13T09:09:16.086560+0000","last_clean":"2024-09-13T09:09:16.086560+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:09:16.086560+0000","last_undegraded":"2024-09-13T09:09:16.086560+0000","last_fullsized":"2024-09-13T09:09:16.086560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"117'540","reported_seq":5031,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.084829+0000","last_change":"2024-09-13T09:08:57.963388+0000","last_active":"2024-09-13T09:09:15.084829+0000","last_peered":"2024-09-13T09:09:15.084829+0000","last_clean":"2024-09-13T09:09:15.084829+0000","last_became_active":"2024-09-13T09:08:57.961197+0000","last_became_peered":"2024-09-13T09:08:57.961197+0000","last_unstale":"2024-09-13T09:09:15.084829+0000","last_undegraded":"2024-09-13T09:09:15.084829+0000","last_fullsized":"2024-09-13T09:09:15.084829+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:00.864515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":50168845,"num_objects":42,"num_object_clones":7,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":5729,"num_read_kb":20326,"num_write":703,"num_write_kb":6584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":6759140,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":122,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.085063+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:09:15.085063+0000","last_peered":"2024-09-13T09:09:15.085063+0000","last_clean":"2024-09-13T09:09:15.085063+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:09:15.085063+0000","last_undegraded":"2024-09-13T09:09:15.085063+0000","last_fullsized":"2024-09-13T09:09:15.085063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"117'338","reported_seq":1999,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T09:09:14.941933+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:09:14.941933+0000","last_peered":"2024-09-13T09:09:14.941933+0000","last_clean":"2024-09-13T09:09:14.941933+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:09:14.941933+0000","last_undegraded":"2024-09-13T09:09:14.941933+0000","last_fullsized":"2024-09-13T09:09:14.941933+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":24424736,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":18,"num_read":2427,"num_read_kb":13487,"num_write":323,"num_write_kb":5128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":115,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.081984+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:09:15.081984+0000","last_peered":"2024-09-13T09:09:15.081984+0000","last_clean":"2024-09-13T09:09:15.081984+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:09:15.081984+0000","last_undegraded":"2024-09-13T09:09:15.081984+0000","last_fullsized":"2024-09-13T09:09:15.081984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"114'417","reported_seq":5264,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T09:09:13.073016+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:09:13.073016+0000","last_peered":"2024-09-13T09:09:13.073016+0000","last_clean":"2024-09-13T09:09:13.073016+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:09:13.073016+0000","last_undegraded":"2024-09-13T09:09:13.073016+0000","last_fullsized":"2024-09-13T09:09:13.073016+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":48989797,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":5767,"num_read_kb":20110,"num_write":563,"num_write_kb":4922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":112,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T09:09:12.058670+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:09:12.058670+0000","last_peered":"2024-09-13T09:09:12.058670+0000","last_clean":"2024-09-13T09:09:12.058670+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:09:12.058670+0000","last_undegraded":"2024-09-13T09:09:12.058670+0000","last_fullsized":"2024-09-13T09:09:12.058670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"117'550","reported_seq":2777,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.079115+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:09:15.079115+0000","last_peered":"2024-09-13T09:09:15.079115+0000","last_clean":"2024-09-13T09:09:15.079115+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:09:15.079115+0000","last_undegraded":"2024-09-13T09:09:15.079115+0000","last_fullsized":"2024-09-13T09:09:15.079115+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":41585035,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":3792,"num_read_kb":16451,"num_write":760,"num_write_kb":10345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"117'351","reported_seq":2531,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.078784+0000","last_change":"2024-09-13T09:08:38.211675+0000","last_active":"2024-09-13T09:09:15.078784+0000","last_peered":"2024-09-13T09:09:15.078784+0000","last_clean":"2024-09-13T09:09:15.078784+0000","last_became_active":"2024-09-13T09:08:38.209914+0000","last_became_peered":"2024-09-13T09:08:38.209914+0000","last_unstale":"2024-09-13T09:09:15.078784+0000","last_undegraded":"2024-09-13T09:09:15.078784+0000","last_fullsized":"2024-09-13T09:09:15.078784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":351,"log_dups_size":0,"ondisk_log_size":351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:19.947594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":78970947,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":2695,"num_read_kb":15262,"num_write":378,"num_write_kb":5445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":918446,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":117,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.081661+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:09:15.081661+0000","last_peered":"2024-09-13T09:09:15.081661+0000","last_clean":"2024-09-13T09:09:15.081661+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:09:15.081661+0000","last_undegraded":"2024-09-13T09:09:15.081661+0000","last_fullsized":"2024-09-13T09:09:15.081661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"114'370","reported_seq":2334,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T09:09:13.068917+0000","last_change":"2024-09-13T09:08:57.959945+0000","last_active":"2024-09-13T09:09:13.068917+0000","last_peered":"2024-09-13T09:09:13.068917+0000","last_clean":"2024-09-13T09:09:13.068917+0000","last_became_active":"2024-09-13T09:08:57.957875+0000","last_became_peered":"2024-09-13T09:08:57.957875+0000","last_unstale":"2024-09-13T09:09:13.068917+0000","last_undegraded":"2024-09-13T09:09:13.068917+0000","last_fullsized":"2024-09-13T09:09:13.068917+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.313991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":25007723,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":25,"num_read":2516,"num_read_kb":12517,"num_write":413,"num_write_kb":9262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3111448,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T09:09:12.058195+0000","last_change":"2024-09-13T09:08:54.932713+0000","last_active":"2024-09-13T09:09:12.058195+0000","last_peered":"2024-09-13T09:09:12.058195+0000","last_clean":"2024-09-13T09:09:12.058195+0000","last_became_active":"2024-09-13T09:08:54.931779+0000","last_became_peered":"2024-09-13T09:08:54.931779+0000","last_unstale":"2024-09-13T09:09:12.058195+0000","last_undegraded":"2024-09-13T09:09:12.058195+0000","last_fullsized":"2024-09-13T09:09:12.058195+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:42:37.917578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"114'320","reported_seq":1704,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-13T09:09:13.072887+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:09:13.072887+0000","last_peered":"2024-09-13T09:09:13.072887+0000","last_clean":"2024-09-13T09:09:13.072887+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:09:13.072887+0000","last_undegraded":"2024-09-13T09:09:13.072887+0000","last_fullsized":"2024-09-13T09:09:13.072887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":15265327,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":1677,"num_read_kb":12812,"num_write":334,"num_write_kb":6737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":115,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.081592+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:09:15.081592+0000","last_peered":"2024-09-13T09:09:15.081592+0000","last_clean":"2024-09-13T09:09:15.081592+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:09:15.081592+0000","last_undegraded":"2024-09-13T09:09:15.081592+0000","last_fullsized":"2024-09-13T09:09:15.081592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"117'327","reported_seq":1938,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T09:09:14.947583+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:09:14.947583+0000","last_peered":"2024-09-13T09:09:14.947583+0000","last_clean":"2024-09-13T09:09:14.947583+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:09:14.947583+0000","last_undegraded":"2024-09-13T09:09:14.947583+0000","last_fullsized":"2024-09-13T09:09:14.947583+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":16147545,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":2135,"num_read_kb":9749,"num_write":360,"num_write_kb":2849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":116,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.081849+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:09:15.081849+0000","last_peered":"2024-09-13T09:09:15.081849+0000","last_clean":"2024-09-13T09:09:15.081849+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:09:15.081849+0000","last_undegraded":"2024-09-13T09:09:15.081849+0000","last_fullsized":"2024-09-13T09:09:15.081849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":244,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.078183+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:09:15.078183+0000","last_peered":"2024-09-13T09:09:15.078183+0000","last_clean":"2024-09-13T09:09:15.078183+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:09:15.078183+0000","last_undegraded":"2024-09-13T09:09:15.078183+0000","last_fullsized":"2024-09-13T09:09:15.078183+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"118'682","reported_seq":5081,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.121299+0000","last_change":"2024-09-13T09:08:57.966773+0000","last_active":"2024-09-13T09:09:15.121299+0000","last_peered":"2024-09-13T09:09:15.121299+0000","last_clean":"2024-09-13T09:09:15.121299+0000","last_became_active":"2024-09-13T09:08:57.964033+0000","last_became_peered":"2024-09-13T09:08:57.964033+0000","last_unstale":"2024-09-13T09:09:15.121299+0000","last_undegraded":"2024-09-13T09:09:15.121299+0000","last_fullsized":"2024-09-13T09:09:15.121299+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:54.967175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":56867726,"num_objects":51,"num_object_clones":11,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":21,"num_read":5951,"num_read_kb":22151,"num_write":1021,"num_write_kb":11043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":4982043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"117'454","reported_seq":3015,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.081707+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:09:15.081707+0000","last_peered":"2024-09-13T09:09:15.081707+0000","last_clean":"2024-09-13T09:09:15.081707+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:09:15.081707+0000","last_undegraded":"2024-09-13T09:09:15.081707+0000","last_fullsized":"2024-09-13T09:09:15.081707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":47878367,"num_objects":40,"num_object_clones":6,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":11,"num_read":4247,"num_read_kb":16076,"num_write":564,"num_write_kb":8838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"117'470","reported_seq":2460,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T09:09:14.601072+0000","last_change":"2024-09-13T09:08:21.089403+0000","last_active":"2024-09-13T09:09:14.601072+0000","last_peered":"2024-09-13T09:09:14.601072+0000","last_clean":"2024-09-13T09:09:14.601072+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:09:14.601072+0000","last_undegraded":"2024-09-13T09:09:14.601072+0000","last_fullsized":"2024-09-13T09:09:14.601072+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":83199091,"num_objects":41,"num_object_clones":12,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":3510,"num_read_kb":21677,"num_write":459,"num_write_kb":16918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"117'328","reported_seq":3026,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T09:09:14.251838+0000","last_change":"2024-09-13T09:08:57.963838+0000","last_active":"2024-09-13T09:09:14.251838+0000","last_peered":"2024-09-13T09:09:14.251838+0000","last_clean":"2024-09-13T09:09:14.251838+0000","last_became_active":"2024-09-13T09:08:57.961707+0000","last_became_peered":"2024-09-13T09:08:57.961707+0000","last_unstale":"2024-09-13T09:09:14.251838+0000","last_undegraded":"2024-09-13T09:09:14.251838+0000","last_fullsized":"2024-09-13T09:09:14.251838+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:58.156711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":37304628,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":17,"num_read":3067,"num_read_kb":30293,"num_write":358,"num_write_kb":16217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":8453655,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"117'259","reported_seq":16544,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.081730+0000","last_change":"2024-09-13T09:08:57.961063+0000","last_active":"2024-09-13T09:09:15.081730+0000","last_peered":"2024-09-13T09:09:15.081730+0000","last_clean":"2024-09-13T09:09:15.081730+0000","last_became_active":"2024-09-13T09:08:57.959990+0000","last_became_peered":"2024-09-13T09:08:57.959990+0000","last_unstale":"2024-09-13T09:09:15.081730+0000","last_undegraded":"2024-09-13T09:09:15.081730+0000","last_fullsized":"2024-09-13T09:09:15.081730+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:21.370554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":50389958,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":20,"num_read":16247,"num_read_kb":26790,"num_write":290,"num_write_kb":14218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":26451574,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"119'303","reported_seq":2609,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T09:09:16.139913+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:09:16.139913+0000","last_peered":"2024-09-13T09:09:16.139913+0000","last_clean":"2024-09-13T09:09:16.139913+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:09:16.139913+0000","last_undegraded":"2024-09-13T09:09:16.139913+0000","last_fullsized":"2024-09-13T09:09:16.139913+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":33024803,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":19,"num_read":2793,"num_read_kb":8781,"num_write":298,"num_write_kb":4642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"118'699","reported_seq":6236,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.124862+0000","last_change":"2024-09-13T09:08:38.205741+0000","last_active":"2024-09-13T09:09:15.124862+0000","last_peered":"2024-09-13T09:09:15.124862+0000","last_clean":"2024-09-13T09:09:15.124862+0000","last_became_active":"2024-09-13T09:08:38.205194+0000","last_became_peered":"2024-09-13T09:08:38.205194+0000","last_unstale":"2024-09-13T09:09:15.124862+0000","last_undegraded":"2024-09-13T09:09:15.124862+0000","last_fullsized":"2024-09-13T09:09:15.124862+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:14.407799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":50292853,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":18,"num_read":8285,"num_read_kb":21727,"num_write":1084,"num_write_kb":4615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"119'342","reported_seq":4254,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T09:09:16.232024+0000","last_change":"2024-09-13T09:09:16.093748+0000","last_active":"2024-09-13T09:09:16.232024+0000","last_peered":"2024-09-13T09:09:16.232024+0000","last_clean":"2024-09-13T09:09:16.232024+0000","last_became_active":"2024-09-13T09:09:16.091543+0000","last_became_peered":"2024-09-13T09:09:16.091543+0000","last_unstale":"2024-09-13T09:09:16.232024+0000","last_undegraded":"2024-09-13T09:09:16.232024+0000","last_fullsized":"2024-09-13T09:09:16.232024+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:12.463442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":44439956,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":19,"num_read":4201,"num_read_kb":14301,"num_write":328,"num_write_kb":5945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":9699455,"num_keys_recovered":77,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"119'406","reported_seq":2376,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T09:09:16.141814+0000","last_change":"2024-09-13T09:08:09.863895+0000","last_active":"2024-09-13T09:09:16.141814+0000","last_peered":"2024-09-13T09:09:16.141814+0000","last_clean":"2024-09-13T09:09:16.141814+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:09:16.141814+0000","last_undegraded":"2024-09-13T09:09:16.141814+0000","last_fullsized":"2024-09-13T09:09:16.141814+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":61404306,"num_objects":58,"num_object_clones":6,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":20,"num_read":2570,"num_read_kb":16873,"num_write":404,"num_write_kb":9415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"117'786","reported_seq":3302,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T09:09:15.081596+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:09:15.081596+0000","last_peered":"2024-09-13T09:09:15.081596+0000","last_clean":"2024-09-13T09:09:15.081596+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:09:15.081596+0000","last_undegraded":"2024-09-13T09:09:15.081596+0000","last_fullsized":"2024-09-13T09:09:15.081596+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":40629101,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":14,"num_read":5628,"num_read_kb":10524,"num_write":1147,"num_write_kb":4843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"117'583","reported_seq":3710,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T09:09:14.943088+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:09:14.943088+0000","last_peered":"2024-09-13T09:09:14.943088+0000","last_clean":"2024-09-13T09:09:14.943088+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:09:14.943088+0000","last_undegraded":"2024-09-13T09:09:14.943088+0000","last_fullsized":"2024-09-13T09:09:14.943088+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":40999502,"num_objects":43,"num_object_clones":10,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":5137,"num_read_kb":12941,"num_write":869,"num_write_kb":10226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"119'461","reported_seq":3088,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T09:09:16.085722+0000","last_change":"2024-09-13T09:09:04.006383+0000","last_active":"2024-09-13T09:09:16.085722+0000","last_peered":"2024-09-13T09:09:16.085722+0000","last_clean":"2024-09-13T09:09:16.085722+0000","last_became_active":"2024-09-13T09:09:04.005219+0000","last_became_peered":"2024-09-13T09:09:04.005219+0000","last_unstale":"2024-09-13T09:09:16.085722+0000","last_undegraded":"2024-09-13T09:09:16.085722+0000","last_fullsized":"2024-09-13T09:09:16.085722+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:56.239198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":23252193,"num_objects":59,"num_object_clones":14,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":24,"num_read":2849,"num_read_kb":10200,"num_write":522,"num_write_kb":6914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1531015154,"num_objects":1572,"num_object_clones":341,"num_object_copies":3144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1572,"num_whiteouts":580,"num_read":183713,"num_read_kb":613888,"num_write":20434,"num_write_kb":313430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1655,"num_bytes_recovered":167543114,"num_keys_recovered":370,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385126400,"data_stored":421263987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16136,"ondisk_log_size":16136,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509471,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148736,"kb_used_data":77424,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94223104,"statfs":{"total":96636764160,"available":96484458496,"internally_reserved":0,"allocated":79282176,"data_stored":89327284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":73005512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.435},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.115,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":1.1459999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.27200000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.121,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.071,"5min":1.444,"15min":1.444},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59399999999999997,"15min":0.57199999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85199999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2989999999999999,"5min":1.343,"15min":1.343},"last":0.622}]},{"osd":5,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.873,"15min":0.86499999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":1.3460000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.92200000000000004,"15min":0.90600000000000003},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.92500000000000004,"15min":0.92100000000000004},"min":{"1min":0.52600000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.70599999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":125052,"kb_used_data":53744,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94246788,"statfs":{"total":96636764160,"available":96508710912,"internally_reserved":0,"allocated":55033856,"data_stored":56564647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73999999999999999,"15min":0.751},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80600000000000005,"15min":0.79900000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.52300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.05},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.88100000000000001,"15min":0.91400000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.0109999999999999,"15min":1.0389999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":1.093},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.95499999999999996},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.492,"5min":1.651,"15min":1.651},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.98299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.92800000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3049999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.012},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.107,"15min":1.131},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5840000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.129}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":0.875}]}]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80624,"kb_used_data":53600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94291216,"statfs":{"total":96636764160,"available":96554205184,"internally_reserved":0,"allocated":54886400,"data_stored":59657234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.625,"15min":0.64400000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.995,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.036,"5min":1.214,"15min":1.214},"last":1.0329999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96899999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.407,"15min":1.407},"last":0.79000000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.121,"5min":1.415,"15min":1.415},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.92200000000000004,"15min":0.94199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2270000000000001,"5min":1.448,"15min":1.448},"last":1.224}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96599999999999997,"15min":0.98599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.96299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2010000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.47399999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.353,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.996},{"interface":"front","average":{"1min":0.878,"5min":1.0309999999999999,"15min":1.0569999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3460000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.167}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.107},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.28}]}]},{"osd":0,"up_from":0,"seq":64424509472,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542175,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65716,"kb_used_data":34604,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94306124,"statfs":{"total":96636764160,"available":96569470976,"internally_reserved":0,"allocated":35434496,"data_stored":39549389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77700000000000002,"15min":0.76400000000000001},"min":{"1min":0.47499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.77500000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.82199999999999995,"15min":0.79700000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":1.012},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.92900000000000005,"15min":0.92800000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":1.2669999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":0.94899999999999995,"15min":0.90100000000000002},"min":{"1min":0.53700000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":8.3499999999999996,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.206,"5min":0.98199999999999998,"15min":0.94399999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":8.2989999999999995,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":1.107}]},{"osd":3,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.95099999999999996,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.472,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.92300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.97299999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.218,"5min":1.7,"15min":1.7},"last":0.88500000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.085,"15min":1.093},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0700000000000001,"15min":1.0720000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.91300000000000003}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.754}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102580,"kb_used_data":67340,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94269260,"statfs":{"total":96636764160,"available":96531722240,"internally_reserved":0,"allocated":68956160,"data_stored":74914236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.84299999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.115,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.202},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":1.3,"15min":1.3},"last":0.97099999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.84799999999999998,"15min":0.81999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.90900000000000003,"15min":0.91600000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.0589999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.873,"15min":0.85299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.585},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.2609999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.615},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.88900000000000001,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.73}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.008,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.7689999999999999},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.849}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":114228,"kb_used_data":60920,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94257612,"statfs":{"total":96636764160,"available":96519794688,"internally_reserved":0,"allocated":62382080,"data_stored":63619834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":1.1200000000000001,"15min":1.179},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0489999999999999,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.82599999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.085,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.218,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.95499999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.1080000000000001,"15min":1.159},"min":{"1min":0.54900000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.151,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":1.042},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.2}]},{"osd":3,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4910000000000001,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.96999999999999997,"15min":0.97799999999999998},"min":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.585},"last":0.871}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0169999999999999,"15min":1.0309999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.357,"15min":1.415},"min":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3799999999999999,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.012,"5min":1.0800000000000001,"15min":1.091},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4710000000000001,"5min":1.899,"15min":1.899},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0720000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6295552,"data_stored":7615357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50098176,"data_stored":54579898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53645312,"data_stored":55921018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":78008320,"data_stored":88790617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67567616,"data_stored":74265527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":60424192,"data_stored":62392574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":33959936,"data_stored":38830984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":35127296,"data_stored":38868012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:09:17.947 INFO:tasks.thrashosds.thrasher:Setting 3.14 to [0, 5, 3, 1] 2024-09-13T09:09:17.947 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.14', '0', '5', '3', '1'] 2024-09-13T09:09:17.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.14 0 5 3 1 2024-09-13T09:09:17.955 INFO:teuthology.orchestra.run.smithi031.stdout:5789 trunc from 0x301b666 to 0xbe7caad 2024-09-13T09:09:17.956 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:17.952+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:17.963 INFO:teuthology.orchestra.run.smithi031.stdout:5790 read 0x853ab0b thru 0x8540e5a (0x6350 bytes) 2024-09-13T09:09:17.963 INFO:teuthology.orchestra.run.smithi031.stdout:5793 read 0x21a824e thru 0x21b6a42 (0xe7f5 bytes) 2024-09-13T09:09:17.967 INFO:teuthology.orchestra.run.smithi031.stdout:5794 trunc from 0xbe7caad to 0xe12ce63 2024-09-13T09:09:17.976 INFO:teuthology.orchestra.run.smithi031.stdout:5795 write 0xb2e47a3 thru 0xb2edd63 (0x95c1 bytes) 2024-09-13T09:09:17.984 INFO:teuthology.orchestra.run.smithi031.stdout:5796 clone 45 order 18 su 65536 sc 9 2024-09-13T09:09:18.020 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:18.012+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:18.045 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:18.045 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:18.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:18.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:09:18.144 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:18.140+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:18.188 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone44 from 0xe12ce63 (overlap 0x301b666) to 0x1cdd3d6 2024-09-13T09:09:18.222 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:18.218+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:18.253 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:18.253 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:18.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:18.323 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:18.318+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:18.363 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-13T09:09:18.424 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:18.422+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:18.444 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:18.445 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:18.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:18.526 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:18.520+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:18.582 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:18.582 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:18.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:18.647 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:18.646+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:18.663 INFO:teuthology.orchestra.run.smithi031.stdout:5797 write 0xe74346d thru 0xe7469c3 (0x3557 bytes) 2024-09-13T09:09:18.668 INFO:teuthology.orchestra.run.smithi031.stdout:5798 writesame 0x9e757a0 thru 0x9e84dc4 data_size 0x1(0xf625 bytes) 2024-09-13T09:09:18.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:18.727 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:18.720+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:18.738 INFO:teuthology.orchestra.run.smithi031.stdout:5800 read 0x8026cd4 thru 0x802e226 (0x7553 bytes) 2024-09-13T09:09:18.738 INFO:teuthology.orchestra.run.smithi031.stdout:5801 read 0xd5eff16 thru 0xd5f1238 (0x1323 bytes) 2024-09-13T09:09:18.738 INFO:teuthology.orchestra.run.smithi031.stdout:5802 trunc from 0xe7469c4 to 0xea9b75c 2024-09-13T09:09:18.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:5803 read 0x72953e5 thru 0x729bced (0x6909 bytes) 2024-09-13T09:09:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:5804 punch from 0x6737d3c to 0x674265a, (0xa91e bytes) 2024-09-13T09:09:18.751 INFO:teuthology.orchestra.run.smithi031.stdout:5805 read 0xaf608a4 thru 0xaf6652e (0x5c8b bytes) 2024-09-13T09:09:18.758 INFO:teuthology.orchestra.run.smithi031.stdout:5806 trunc from 0xea9b75c to 0x9b713c8 2024-09-13T09:09:18.790 INFO:teuthology.orchestra.run.smithi031.stdout:5808 writesame 0x737ad96 thru 0x737e974 data_size 0xd(0x3bdf bytes) 2024-09-13T09:09:18.797 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:18.797 INFO:teuthology.orchestra.run.smithi031.stdout:5811 read 0xa2750 thru 0xab401 (0x8cb2 bytes) 2024-09-13T09:09:18.799 INFO:teuthology.orchestra.run.smithi031.stdout:5812 write 0x1dc4f29 thru 0x1dcdf01 (0x8fd9 bytes) 2024-09-13T09:09:18.809 INFO:teuthology.orchestra.run.smithi031.stdout:5813 writesame 0xc1c447 thru 0xc25131 data_size 0x19(0x8ceb bytes) 2024-09-13T09:09:18.816 INFO:teuthology.orchestra.run.smithi031.stdout:5815 write 0x1b71668 thru 0x1b7d027 (0xb9c0 bytes) 2024-09-13T09:09:18.822 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:18.822 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:18.828 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:18.820+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:18.838 INFO:teuthology.orchestra.run.smithi031.stdout:5816 write 0xc2bd664 thru 0xc2cc1c1 (0xeb5e bytes) 2024-09-13T09:09:18.841 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:18.841 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:18.844 INFO:teuthology.orchestra.run.smithi031.stdout:5818 read 0x63080ae thru 0x6313474 (0xb3c7 bytes) 2024-09-13T09:09:18.844 INFO:teuthology.orchestra.run.smithi031.stdout:5819 write 0x7beb4c8 thru 0x7bf8281 (0xcdba bytes) 2024-09-13T09:09:18.851 INFO:teuthology.orchestra.run.smithi031.stdout:5820 writesame 0x2bc6755 thru 0x2bd6605 data_size 0x1(0xfeb1 bytes) 2024-09-13T09:09:18.863 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:18.863 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:18.873 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:18.874 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:18.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:18.874 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:18.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:18.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.759186060999994, 2024-09-13T09:09:18.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:09:18.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.759165293999999, 2024-09-13T09:09:18.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:18.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:09:18.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:18.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:09:18.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:09:18.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.759147442, 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:09:18.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:09:18.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:09:18.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.759130876, 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:09:18.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.896 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.896 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:18.896 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.759105290999997, 2024-09-13T09:09:18.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:09:18.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.759086124000007, 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:18.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.759068841000001, 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:09:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:09:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:18.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.758539971000005, 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:09:18.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.758508066000005, 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:09:18.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:09:18.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:09:18.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.758483118000001, 2024-09-13T09:09:18.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:18.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.758178552999993, 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.756416130000005, 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:18.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:09:18.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.756395280000007, 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:09:18.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.756372956000007, 2024-09-13T09:09:18.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:18.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:09:18.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.756354267000006, 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:09:18.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.952 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.756337215000002, 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:09:18.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.756320191, 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:09:18.961 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.746307204000004, 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:18.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:09:18.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.366152681000003, 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:18.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.365972259000003, 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:18.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:18.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:18.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:18.976 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:18.977 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:18.978 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:18.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:18.979 INFO:teuthology.orchestra.run.smithi031.stdout:5821 read 0x834bbbd thru 0x834fdec (0x4230 bytes) 2024-09-13T09:09:18.979 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:18.979 INFO:teuthology.orchestra.run.smithi031.stdout:5824 write 0x82a11fa thru 0x82a78cf (0x66d6 bytes) 2024-09-13T09:09:18.980 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:18.980 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:18.981 INFO:teuthology.orchestra.run.smithi031.stdout:5825 punch from 0xbfc5c3c to 0xbfd4405, (0xe7c9 bytes) 2024-09-13T09:09:18.981 INFO:teuthology.orchestra.run.smithi031.stdout:5826 read 0xa511009 thru 0xa51e492 (0xd48a bytes) 2024-09-13T09:09:18.981 INFO:teuthology.orchestra.run.smithi031.stdout:5828 read 0xc261ed2 thru 0xc2640ca (0x21f9 bytes) 2024-09-13T09:09:18.981 INFO:teuthology.orchestra.run.smithi031.stdout:5829 read 0x4fbe38f thru 0x4fbee02 (0xa74 bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5830 punch from 0x68171c2 to 0x681897e, (0x17bc bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5831 punch from 0x4125669 to 0x412ff6d, (0xa904 bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5832 read 0x6189339 thru 0x618d1c3 (0x3e8b bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5833 read 0x8b096f5 thru 0x8b106fc (0x7008 bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5837 write 0x57dc744 thru 0x57e511d (0x89da bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5838 write 0x2a8a046 thru 0x2a8ef96 (0x4f51 bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5839 writesame 0xbaaeac4 thru 0xbabd6fc data_size 0x35(0xec39 bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5842 write 0x6969e2d thru 0x6978e22 (0xeff6 bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5843 write 0x49cffed thru 0x49ddb03 (0xdb17 bytes) 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5844 trunc from 0xc2cc1c2 to 0xad76c53 2024-09-13T09:09:18.982 INFO:teuthology.orchestra.run.smithi031.stdout:5845 write 0xe7ce9fe thru 0xe7d3c1e (0x5221 bytes) 2024-09-13T09:09:18.983 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:18.974+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:18.983 INFO:teuthology.orchestra.run.smithi031.stdout:5846 writesame 0x77235e3 thru 0x772dd07 data_size 0x11(0xa725 bytes) 2024-09-13T09:09:18.987 INFO:teuthology.orchestra.run.smithi031.stdout:5847 flatten 2024-09-13T09:09:19.078 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:19.074+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:19.113 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:09:19.116 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.14 pg_upmap_items mapping to [0->5,3->1] 2024-09-13T09:09:19.128 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154260 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.019317109999999998, 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019341812999999999, 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154260 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114113+0000", 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114117+0000", 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3139999999999999e-06 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114125+0000", 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1429999999999998e-06 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114131+0000", 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119237+0000", 2024-09-13T09:09:19.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051056859999999999 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119239+0000", 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154271 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:19.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.019294655000000001, 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019365822000000001, 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154271 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114138+0000", 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114145+0000", 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114150+0000", 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T09:09:19.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119251+0000", 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051011809999999998 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119256+0000", 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154272 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.019276976000000001, 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019385478000000001, 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154272 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:19.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114156+0000", 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114162+0000", 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114167+0000", 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.977e-06 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119274+0000", 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051065360000000001 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119279+0000", 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:09:19.148 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154273 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.019259132000000002, 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019402686999999998, 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154273 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114171+0000", 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114174+0000", 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114181+0000", 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5030000000000002e-06 2024-09-13T09:09:19.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114185+0000", 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6720000000000003e-06 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119297+0000", 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051116549999999997 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119302+0000", 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154274 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.019242832000000001, 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019422762999999999, 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154274 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:19.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114190+0000", 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7519999999999999e-06 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114193+0000", 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114195+0000", 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.407e-06 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119320+0000", 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051245739999999998 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119323+0000", 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154293 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.016214528999999998, 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016429671999999999, 2024-09-13T09:09:19.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154293 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117215+0000", 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117223+0000", 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:09:19.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117236+0000", 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3492000000000001e-05 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117247+0000", 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0725e-05 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119334+0000", 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002087392 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119337+0000", 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154208 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.022530768, 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022782178, 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154208 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110898+0000", 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110909+0000", 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0896999999999999e-05 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110932+0000", 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3544999999999999e-05 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110943+0000", 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.064e-05 2024-09-13T09:09:19.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119027+0000", 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080840670000000003 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119033+0000", 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0050000000000001e-06 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154209 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.022472710999999999, 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022760807000000001, 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154209 2024-09-13T09:09:19.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110961+0000", 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110967+0000", 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4470000000000003e-06 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110972+0000", 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119044+0000", 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080713150000000008 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119046+0000", 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-13T09:09:19.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154218 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.022396830999999999, 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.02271985, 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154218 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111033+0000", 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111038+0000", 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111047+0000", 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1779999999999992e-06 2024-09-13T09:09:19.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111053+0000", 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9089999999999997e-06 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119060+0000", 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080075760000000006 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119065+0000", 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154219 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.022355051000000001, 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022713152, 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154219 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111075+0000", 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111079+0000", 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111086+0000", 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111090+0000", 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119079+0000", 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007988396 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119081+0000", 2024-09-13T09:09:19.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154228 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.022014695000000001, 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022407633, 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.178 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154228 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111415+0000", 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111421+0000", 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111432+0000", 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1807000000000001e-05 2024-09-13T09:09:19.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111439+0000", 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8909999999999998e-06 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119092+0000", 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076528159999999998 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119097+0000", 2024-09-13T09:09:19.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154229 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.02124908, 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021677132000000002, 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.182 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154229 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112181+0000", 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112186+0000", 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249999999999997e-06 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112202+0000", 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.543e-05 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112209+0000", 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5669999999999997e-06 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119115+0000", 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069053999999999999 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119120+0000", 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-13T09:09:19.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154238 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.021214769000000001, 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021677348999999999, 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154238 2024-09-13T09:09:19.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:19.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112219+0000", 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0240000000000002e-06 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112225+0000", 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112230+0000", 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119130+0000", 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068996450000000003 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119132+0000", 2024-09-13T09:09:19.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154239 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.021196295, 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021693328000000001, 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154239 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112234+0000", 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112237+0000", 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112243+0000", 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112247+0000", 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119149+0000", 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069020679999999999 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119154+0000", 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154242 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.021176892999999999, 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021712121000000001, 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154242 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112256+0000", 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112262+0000", 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112267+0000", 2024-09-13T09:09:19.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119166+0000", 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068991599999999997 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119168+0000", 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.678e-06 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154243 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.021149041, 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021719196, 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154243 2024-09-13T09:09:19.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112281+0000", 2024-09-13T09:09:19.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112285+0000", 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112292+0000", 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.126e-06 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112297+0000", 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0760000000000002e-06 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119178+0000", 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068802890000000004 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119180+0000", 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154244 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.021128892999999999, 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021733764999999999, 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154244 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112304+0000", 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112310+0000", 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1859999999999997e-06 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112315+0000", 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:09:19.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119189+0000", 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068746950000000001 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119192+0000", 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154251 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.020530502999999999, 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021170119000000001, 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154251 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112899+0000", 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:19.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112905+0000", 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5709999999999997e-06 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112916+0000", 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112923+0000", 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119201+0000", 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062780379999999997 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119204+0000", 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154252 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.019389079, 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020063613000000001, 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154252 2024-09-13T09:09:19.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114041+0000", 2024-09-13T09:09:19.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:19.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114046+0000", 2024-09-13T09:09:19.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T09:09:19.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114062+0000", 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5951999999999999e-05 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114070+0000", 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9510000000000006e-06 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119213+0000", 2024-09-13T09:09:19.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051428860000000002 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119215+0000", 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4509999999999999e-06 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154253 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.019353868999999999, 2024-09-13T09:09:19.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020062930999999999, 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154253 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114080+0000", 2024-09-13T09:09:19.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114087+0000", 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1679999999999998e-06 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114092+0000", 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119225+0000", 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051325920000000001 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119227+0000", 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 20 2024-09-13T09:09:19.218 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:19.220 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:19.221 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:19.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:19.222 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:19.223 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:19.223 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:19.212+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:19.321 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:19.316+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:19.390 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:19.390 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:19.390 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:19.390 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:19.390 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:19.406 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:19.422 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:19.420+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:19.523 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:19.518+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39047 3.10 3:0920097a:::rbd_data.1124d4e7fcc5.0000000000000030:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 86.580498249000001, 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90968864100000002, 2024-09-13T09:09:19.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39047 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952527+0000", 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952523+0000", 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999967 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952537+0000", 2024-09-13T09:09:19.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3772e-05 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952563+0000", 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5964000000000001e-05 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952577+0000", 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4109e-05 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952644+0000", 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7531000000000001e-05 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.952652+0000", 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.055e-06 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862136+0000", 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90948311800000003 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.862215+0000", 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9647999999999994e-05 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88209 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24748032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:19.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297948657000006, 2024-09-13T09:09:19.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033308870999999997, 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88209 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:19.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235076+0000", 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235077+0000", 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.22e-06 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235080+0000", 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1130000000000001e-06 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235083+0000", 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3819999999999998e-06 2024-09-13T09:09:19.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268290+0000", 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033207532999999997 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268342+0000", 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1959999999999997e-05 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268385+0000", 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2874999999999999e-05 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88210 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24772608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297940186000005, 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033438816000000003, 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88210 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235085+0000", 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235084+0000", 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235086+0000", 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1659999999999999e-06 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235088+0000", 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235091+0000", 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4839999999999998e-06 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268421+0000", 2024-09-13T09:09:19.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033329923999999997 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268479+0000", 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8529999999999997e-05 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268523+0000", 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4048999999999998e-05 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88211 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24797184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297931720999998, 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033554247000000002, 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88211 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235093+0000", 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235094+0000", 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2300000000000001e-06 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235097+0000", 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.021e-06 2024-09-13T09:09:19.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235099+0000", 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3250000000000002e-06 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268550+0000", 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033450510000000003 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268593+0000", 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3035999999999999e-05 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268647+0000", 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4336999999999998e-05 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88212 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24821760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297923333, 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033675415, 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88212 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235101+0000", 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235102+0000", 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.176e-06 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235105+0000", 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9440000000000001e-06 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235108+0000", 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.367e-06 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268686+0000", 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033577810999999999 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268737+0000", 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1134999999999997e-05 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268777+0000", 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0187e-05 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88213 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24846336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297914129999995, 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033785911000000002, 2024-09-13T09:09:19.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88213 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235111+0000", 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235110+0000", 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235112+0000", 2024-09-13T09:09:19.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.728e-06 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235119+0000", 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3239999999999997e-06 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235124+0000", 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.352e-06 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268806+0000", 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033682479000000001 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268855+0000", 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8359999999999998e-05 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268897+0000", 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1915000000000001e-05 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88214 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24870912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297896495000003, 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033897243000000001, 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88214 2024-09-13T09:09:19.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235128+0000", 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235130+0000", 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3920000000000001e-06 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235139+0000", 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3250000000000008e-06 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235151+0000", 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1856e-05 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268933+0000", 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033782874999999997 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.268986+0000", 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2593000000000003e-05 2024-09-13T09:09:19.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269026+0000", 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9546999999999999e-05 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88215 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24895488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297871107999995, 2024-09-13T09:09:19.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033999110999999999, 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88215 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235154+0000", 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235153+0000", 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235155+0000", 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8810000000000001e-06 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235159+0000", 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7500000000000001e-06 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235161+0000", 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3410000000000001e-06 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269056+0000", 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033895001000000001 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269109+0000", 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2417999999999999e-05 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269153+0000", 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3999999999999999e-05 2024-09-13T09:09:19.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88216 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24920064~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297861350000005, 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120667, 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88216 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:19.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235163+0000", 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235173+0000", 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8490000000000001e-06 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235178+0000", 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7219999999999999e-06 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235180+0000", 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.565e-06 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269188+0000", 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034007280000000001 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269245+0000", 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6891000000000001e-05 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269284+0000", 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9561999999999998e-05 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88217 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24944640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297841935999998, 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034230180999999998, 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88217 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235183+0000", 2024-09-13T09:09:19.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235184+0000", 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3120000000000001e-06 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235187+0000", 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9979999999999999e-06 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235192+0000", 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3879999999999999e-06 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269313+0000", 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034120708 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269363+0000", 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9453999999999998e-05 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269413+0000", 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0531000000000001e-05 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88218 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24969216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297830379000004, 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034354619000000003, 2024-09-13T09:09:19.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88218 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235194+0000", 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235195+0000", 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.173e-06 2024-09-13T09:09:19.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235199+0000", 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1370000000000002e-06 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235201+0000", 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3980000000000002e-06 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269450+0000", 2024-09-13T09:09:19.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034248860999999999 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269502+0000", 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2519000000000003e-05 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269549+0000", 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6740999999999999e-05 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88219 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 24993792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297821838000004, 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034466610000000002, 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88219 2024-09-13T09:09:19.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235203+0000", 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235204+0000", 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.184e-06 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235207+0000", 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.813e-06 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235209+0000", 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.182e-06 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269577+0000", 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034368528000000002 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269628+0000", 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0241999999999998e-05 2024-09-13T09:09:19.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269670+0000", 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1875999999999997e-05 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88220 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25018368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297813988000001, 2024-09-13T09:09:19.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034591746999999999, 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88220 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235211+0000", 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235212+0000", 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0839999999999999e-06 2024-09-13T09:09:19.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235215+0000", 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9830000000000001e-06 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235217+0000", 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2879999999999999e-06 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269707+0000", 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034489989999999998 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269762+0000", 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5430000000000003e-05 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269803+0000", 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0181e-05 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88221 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25042944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297805905000004, 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034709489000000003, 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88221 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235219+0000", 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235220+0000", 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0640000000000001e-06 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235223+0000", 2024-09-13T09:09:19.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235225+0000", 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2759999999999999e-06 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269830+0000", 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034604633000000003 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269884+0000", 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4067000000000003e-05 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269928+0000", 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4607000000000002e-05 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88222 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25067520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297797759000005, 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034835487999999998, 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88222 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235227+0000", 2024-09-13T09:09:19.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235228+0000", 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235231+0000", 2024-09-13T09:09:19.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9560000000000002e-06 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235233+0000", 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3199999999999998e-06 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.269963+0000", 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034730039999999997 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270016+0000", 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2581999999999999e-05 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270063+0000", 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6718000000000002e-05 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88223 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25092096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297789538000004, 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034953250999999998, 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88223 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235235+0000", 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235236+0000", 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.127e-06 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235239+0000", 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0180000000000002e-06 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235241+0000", 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.278e-06 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270091+0000", 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034849036 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270137+0000", 2024-09-13T09:09:19.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6112999999999997e-05 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270189+0000", 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88224 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25116672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297781338999997, 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035071738999999998, 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88224 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235243+0000", 2024-09-13T09:09:19.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235244+0000", 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.085e-06 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235247+0000", 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7750000000000001e-06 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235249+0000", 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3190000000000002e-06 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270226+0000", 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.034977019999999998 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270275+0000", 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8791000000000001e-05 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270315+0000", 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9953999999999997e-05 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:88225 3.1c 3:3e5d3684:::rbd_data.1124943a2b90.0000000000000002:16 [read 25141248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.297773426999996, 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035182109000000003, 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.618 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 88225 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235251+0000", 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235252+0000", 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.077e-06 2024-09-13T09:09:19.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235255+0000", 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9610000000000001e-06 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.235258+0000", 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270337+0000", 2024-09-13T09:09:19.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.035079359999999997 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270388+0000", 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0986000000000003e-05 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:05.270433+0000", 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5581000000000003e-05 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93981 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [watch reconnect cookie 140193706156944 gen 8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.678925888999999, 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116345170000001, 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93981 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854099+0000", 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854096+0000", 2024-09-13T09:09:19.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854113+0000", 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6759999999999999e-05 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854146+0000", 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3506000000000001e-05 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.854169+0000", 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2331999999999999e-05 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856384+0000", 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022150960000000002 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.856397+0000", 2024-09-13T09:09:19.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3047999999999999e-05 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863781+0000", 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007384276 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.863918+0000", 2024-09-13T09:09:19.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013628800000000001 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865733+0000", 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018159109999999999 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:93982 3.1a 3:5b46a043:::rbd_header.11245dedf53f:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-13T09:09:19.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.675261957000004, 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0094988519999999, 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 93982 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:19.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857763+0000", 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857761+0000", 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857770+0000", 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2109999999999998e-06 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857789+0000", 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9111999999999999e-05 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857800+0000", 2024-09-13T09:09:19.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1029999999999999e-05 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857854+0000", 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4030999999999998e-05 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:08.857866+0000", 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2275e-05 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864650+0000", 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0067835570000001 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.864768+0000", 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000118019 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.865240+0000", 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00047203799999999998 2024-09-13T09:09:19.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.866250+0000", 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001009935 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867165+0000", 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00091516600000000003 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:19.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867219+0000", 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4330999999999999e-05 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:09.867262+0000", 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.265e-05 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:19.635 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:19.637 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:19.638 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:19.632+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:19.738 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:19.734+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:19.809 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:19.810 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:19.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:19.839 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:19.832+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:19.940 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:19.932+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:19.992 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:19.993 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:19.993 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:20.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:20.041 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:20.036+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:20.135 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:20.135 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:20.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:20.135 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:20.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.135 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.135 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.79197004300001, 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:20.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.79195326999999, 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:09:20.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:09:20.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791915965, 2024-09-13T09:09:20.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:09:20.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791896251, 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:20.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.79187992600001, 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:09:20.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:09:20.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.79186405999999, 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:20.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.79184827500001, 2024-09-13T09:09:20.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:09:20.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:09:20.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791829204, 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:09:20.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:09:20.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.79181265699999, 2024-09-13T09:09:20.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791785058, 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:20.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:09:20.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:09:20.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:09:20.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791767816, 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:09:20.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:09:20.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:20.189 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791737373, 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:09:20.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:09:20.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.79171845800001, 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:09:20.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.79170067, 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:09:20.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791675446, 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:09:20.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791657242, 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:20.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:09:20.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.212 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791636924, 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:09:20.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791618376, 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:20.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791601102, 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:09:20.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:09:20.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 104.791583648, 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:09:20.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:09:20.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.231 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:20.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:20.233 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:20.228+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:20.334 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:20.328+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:20.363 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:20.363 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:20.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:20.435 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:20.432+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:20.503 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:20.503 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:20.504 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:20.504 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:20.504 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:20.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:20.567 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:20.566+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:20.637 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:20.636+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:20.647 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:20.648 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:20.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:20.648 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:20.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:09:20.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:20.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.51324152199999, 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:09:20.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:09:20.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 87.684959341999999, 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:09:20.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:09:20.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:09:20.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.392519504000006, 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:09:20.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:09:20.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.665 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.392439706000005, 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.666 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:09:20.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.392190635000006, 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:09:20.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.391877493999999, 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:09:20.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:09:20.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.391424262000001, 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.391399747999998, 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:09:20.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.391134519000005, 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:09:20.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:09:20.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.390846670000002, 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:09:20.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:09:20.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.390385366000004, 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:20.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:09:20.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.451852017, 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:20.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:09:20.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.440858697000003, 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:09:20.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:09:20.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.436876798999997, 2024-09-13T09:09:20.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:09:20.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.434248805000003, 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:09:20.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:09:20.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:09:20.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.432696020999998, 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:09:20.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.431442705000002, 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:20.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:09:20.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:09:20.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:09:20.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:09:20.743 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.430340672, 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:09:20.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:09:20.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:09:20.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.429130094999998, 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:09:20.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:09:20.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.757 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3:f1cafc87:::rbd_data.1124651f5f97.000000000000004a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 557056~22319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5214346800000005, 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97276468599999999, 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:09:20.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:09:20.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092738+0000", 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97003675 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092877+0000", 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139245 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093029+0000", 2024-09-13T09:09:20.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015224999999999999 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093847+0000", 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081788800000000001 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095250+0000", 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014028529999999999 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095282+0000", 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1961000000000001e-05 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095303+0000", 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:20.764 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:20.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:20.766 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:20.762+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:20.867 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:20.860+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:20.900 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:20.900 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:20.901 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:20.901 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:20.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:20.969 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:20.966+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:21.070 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:21.064+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:21.077 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:21.077 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:21.077 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:21.077 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:21.077 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:21.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:21.171 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:21.170+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:21.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:09:21.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:21.264 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236991644, 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:09:21.265 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:09:21.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:09:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236944311000002, 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.274 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236886775000002, 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:09:21.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:09:21.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236835288999998, 2024-09-13T09:09:21.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:21.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:09:21.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236788347000001, 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.284 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:09:21.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:09:21.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:09:21.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236740953999998, 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:09:21.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236693678999998, 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.294 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:09:21.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236647478999998, 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.299 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:09:21.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:21.303 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.236583474, 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.304 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:09:21.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.308 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.234112385000003, 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:09:21.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.234056663000004, 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:09:21.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:09:21.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.23400676, 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.319 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:09:21.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.233959704, 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:09:21.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:09:21.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.328 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.233911221, 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.233845955999996, 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:09:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.233799039999994, 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:09:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:09:21.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.342 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.233711592000006, 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:09:21.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.232249742999997, 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:21.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.351 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.232171656999995, 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 73.232119568000002, 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:09:21.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:21.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:09:21.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:09:21.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.361 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:21.361 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:21.361 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:21.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:21.363 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:21.358+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:21.397 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:21.398 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:21.410 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:21.411 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:21.427 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:21.427 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:21.444 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:21.444 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:21.463 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:21.463 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:21.464 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:21.456+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:21.481 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:21.481 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:21.495 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:21.495 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:21.500 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:21.501 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:21.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:21.520 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:21.520 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:21.583 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:21.582+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:21.665 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:21.662+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:21.682 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:21.682 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:21.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:21.766 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:21.760+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:21.816 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:21.816 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:21.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:21.867 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:21.866+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:21.968 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:21.960+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:22.003 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:22.003 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:22.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:22.105 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:22.100+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:22.145 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:09:22.169 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:22.164+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:22.212 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:22.212 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:22.223 INFO:teuthology.orchestra.run.smithi031.stdout:5848 trunc from 0xe7d3c1f to 0x6265a81 2024-09-13T09:09:22.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:22.248 INFO:teuthology.orchestra.run.smithi031.stdout:5849 trunc from 0x6265a81 to 0xab84a68 2024-09-13T09:09:22.256 INFO:teuthology.orchestra.run.smithi031.stdout:5850 write 0x84d418a thru 0x84dd7a5 (0x961c bytes) 2024-09-13T09:09:22.260 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:22.260 INFO:teuthology.orchestra.run.smithi031.stdout:5852 write 0x5e8f6b0 thru 0x5e9cb17 (0xd468 bytes) 2024-09-13T09:09:22.267 INFO:teuthology.orchestra.run.smithi031.stdout:5854 punch from 0x9210a16 to 0x921b264, (0xa84e bytes) 2024-09-13T09:09:22.267 INFO:teuthology.orchestra.run.smithi031.stdout:5855 read 0x1a63eb7 thru 0x1a72b19 (0xec63 bytes) 2024-09-13T09:09:22.267 INFO:teuthology.orchestra.run.smithi031.stdout:5856 punch from 0xac67b4 to 0xac86d0, (0x1f1c bytes) 2024-09-13T09:09:22.270 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:22.266+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:22.272 INFO:teuthology.orchestra.run.smithi031.stdout:5857 trunc from 0xab84a68 to 0xde21407 2024-09-13T09:09:22.297 INFO:teuthology.orchestra.run.smithi031.stdout:5859 trunc from 0xde21407 to 0x30ee556 2024-09-13T09:09:22.312 INFO:teuthology.orchestra.run.smithi031.stdout:5860 read 0x42e6a thru 0x46c37 (0x3dce bytes) 2024-09-13T09:09:22.327 INFO:teuthology.orchestra.run.smithi031.stdout:5862 write 0x833b4ec thru 0x834617c (0xac91 bytes) 2024-09-13T09:09:22.333 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:22.353 INFO:teuthology.orchestra.run.smithi031.stdout:5865 write 0xe860120 thru 0xe8669fc (0x68dd bytes) 2024-09-13T09:09:22.357 INFO:teuthology.orchestra.run.smithi031.stdout:5866 writesame 0x4817e73 thru 0x481e3b5 data_size 0x1(0x6543 bytes) 2024-09-13T09:09:22.371 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:22.364+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:22.373 INFO:teuthology.orchestra.run.smithi031.stdout:5867 read 0x6d75cb1 thru 0x6d7fad3 (0x9e23 bytes) 2024-09-13T09:09:22.374 INFO:teuthology.orchestra.run.smithi031.stdout:5868 writesame 0xc1e3d83 thru 0xc1f3498 data_size 0x1(0xf716 bytes) 2024-09-13T09:09:22.399 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:22.399 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:22.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:22.422 INFO:teuthology.orchestra.run.smithi031.stdout:5869 read 0xab996ba thru 0xaba0277 (0x6bbe bytes) 2024-09-13T09:09:22.445 INFO:teuthology.orchestra.run.smithi031.stdout:5870 trunc from 0xe8669fd to 0x1ab1388 2024-09-13T09:09:22.472 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:22.470+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:22.511 INFO:teuthology.orchestra.run.smithi031.stdout:5871 write 0xcca0bad thru 0xcca12e7 (0x73b bytes) 2024-09-13T09:09:22.516 INFO:teuthology.orchestra.run.smithi031.stdout:5872 read 0x1d25447 thru 0x1d2f94a (0xa504 bytes) 2024-09-13T09:09:22.521 INFO:teuthology.orchestra.run.smithi031.stdout:5876 trunc from 0xcca12e8 to 0xee18bcc 2024-09-13T09:09:22.532 INFO:teuthology.orchestra.run.smithi031.stdout:5878 punch from 0xec43bc0 to 0xec468e0, (0x2d20 bytes) 2024-09-13T09:09:22.532 INFO:teuthology.orchestra.run.smithi031.stdout:5879 punch from 0xda29044 to 0xda35935, (0xc8f1 bytes) 2024-09-13T09:09:22.532 INFO:teuthology.orchestra.run.smithi031.stdout:5880 writesame 0x4a078bf thru 0x4a0e527 data_size 0x21(0x6c69 bytes) 2024-09-13T09:09:22.540 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:22.541 INFO:teuthology.orchestra.run.smithi031.stdout:5883 write 0x650bb6f thru 0x65137e1 (0x7c73 bytes) 2024-09-13T09:09:22.574 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:22.568+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:22.588 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:22.588 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:22.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:22.672 INFO:teuthology.orchestra.run.smithi031.stdout:5885 read 0x3106ef7 thru 0x3116680 (0xf78a bytes) 2024-09-13T09:09:22.672 INFO:teuthology.orchestra.run.smithi031.stdout:5888 punch from 0xe0fe34e to 0xe107b34, (0x97e6 bytes) 2024-09-13T09:09:22.672 INFO:teuthology.orchestra.run.smithi031.stdout:5889 read 0xe1b0d31 thru 0xe1b5888 (0x4b58 bytes) 2024-09-13T09:09:22.672 INFO:teuthology.orchestra.run.smithi031.stdout:5890 punch from 0xee0b481 to 0xee0e3f0, (0x2f6f bytes) 2024-09-13T09:09:22.672 INFO:teuthology.orchestra.run.smithi031.stdout:5892 punch from 0xbe23644 to 0xbe29350, (0x5d0c bytes) 2024-09-13T09:09:22.672 INFO:teuthology.orchestra.run.smithi031.stdout:5893 write 0x456b6e8 thru 0x4573b7c (0x8495 bytes) 2024-09-13T09:09:22.674 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:22.670+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:22.681 INFO:teuthology.orchestra.run.smithi031.stdout:5894 write 0x27eb81a thru 0x27f16dc (0x5ec3 bytes) 2024-09-13T09:09:22.690 INFO:teuthology.orchestra.run.smithi031.stdout:5895 writesame 0x40c4af9 thru 0x40c7430 data_size 0x4(0x2938 bytes) 2024-09-13T09:09:22.699 INFO:teuthology.orchestra.run.smithi031.stdout:5897 clone 46 order 19 su 16384 sc 13 2024-09-13T09:09:22.774 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:22.775 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:22.775 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:22.774+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:22.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:22.876 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:22.872+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:22.913 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:22.927 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:22.977 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:22.974+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:23.063 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:23.063 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:23.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:23.078 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:23.074+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:23.179 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:23.174+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:23.260 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:23.260 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:23.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:23.280 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:23.272+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:23.381 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:23.378+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:23.458 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:23.459 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:23.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:23.488 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-13T09:09:23.488 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:23.486+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:23.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:23.583 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:23.582+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:23.625 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:23.625 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:23.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:23.683 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:23.682+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:23.686 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:23.686 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:23.702 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:23.703 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:23.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:23.719 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:23.735 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:23.735 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:23.755 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:23.755 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:23.775 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:23.776 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:23.784 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:23.780+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:23.796 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:23.797 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:23.797 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:23.797 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:23.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:23.818 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:23.818 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:23.885 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:23.880+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:23.937 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154208 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8243250880000002, 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019831428, 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154208 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:23.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110898+0000", 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110909+0000", 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0896999999999999e-05 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110932+0000", 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3544999999999999e-05 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110943+0000", 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.064e-05 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119027+0000", 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080840670000000003 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119033+0000", 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0050000000000001e-06 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130694+0000", 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116608949999999 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130732+0000", 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7696e-05 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154209 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8242670309999998, 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199906329999999, 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154209 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110961+0000", 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:09:23.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110967+0000", 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4470000000000003e-06 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110972+0000", 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119044+0000", 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080713150000000008 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119046+0000", 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130938+0000", 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0118917460000001 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130949+0000", 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0607e-05 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154218 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.824191151, 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019944518, 2024-09-13T09:09:23.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154218 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111033+0000", 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111038+0000", 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-13T09:09:23.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111047+0000", 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1779999999999992e-06 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111053+0000", 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9089999999999997e-06 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119060+0000", 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080075760000000006 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119065+0000", 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130971+0000", 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119057490000001 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130979+0000", 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8790000000000003e-06 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154219 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8241493709999999, 2024-09-13T09:09:23.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199227849999999, 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154219 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111075+0000", 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111079+0000", 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111086+0000", 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111090+0000", 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-13T09:09:23.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119079+0000", 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007988396 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119081+0000", 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130995+0000", 2024-09-13T09:09:23.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011913643 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130999+0000", 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6940000000000001e-06 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154228 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:23.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8238090150000001, 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019595799, 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154228 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111415+0000", 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111421+0000", 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111432+0000", 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1807000000000001e-05 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111439+0000", 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8909999999999998e-06 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119092+0000", 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076528159999999998 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119097+0000", 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131008+0000", 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011911539 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131012+0000", 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5760000000000002e-06 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154229 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8230433999999995, 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01884818, 2024-09-13T09:09:23.963 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154229 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112181+0000", 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112186+0000", 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249999999999997e-06 2024-09-13T09:09:23.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112202+0000", 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.543e-05 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112209+0000", 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5669999999999997e-06 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119115+0000", 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069053999999999999 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119120+0000", 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131023+0000", 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011903446 2024-09-13T09:09:23.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131030+0000", 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154238 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:23.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8230090890000001, 2024-09-13T09:09:23.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188343550000001, 2024-09-13T09:09:23.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154238 2024-09-13T09:09:23.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.970 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:23.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112219+0000", 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0240000000000002e-06 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112225+0000", 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112230+0000", 2024-09-13T09:09:23.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119130+0000", 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068996450000000003 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119132+0000", 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:09:23.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131046+0000", 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119135260000001 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131050+0000", 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5859999999999998e-06 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154239 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8229906150000001, 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188390510000001, 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154239 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112234+0000", 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112237+0000", 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112243+0000", 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-13T09:09:23.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112247+0000", 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119149+0000", 2024-09-13T09:09:23.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069020679999999999 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119154+0000", 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131067+0000", 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011912659 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131074+0000", 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9709999999999998e-06 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154242 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:23.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8229712129999998, 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188413620000001, 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154242 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112256+0000", 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112262+0000", 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112267+0000", 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119166+0000", 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068991599999999997 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119168+0000", 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.678e-06 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131088+0000", 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011919435 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131095+0000", 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5769999999999996e-06 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154243 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8229433610000001, 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018836657, 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154243 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112281+0000", 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112285+0000", 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:09:23.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112292+0000", 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.126e-06 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112297+0000", 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0760000000000002e-06 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119178+0000", 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068802890000000004 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119180+0000", 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131111+0000", 2024-09-13T09:09:23.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011930947 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131119+0000", 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154244 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8229232130000002, 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188321789999999, 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154244 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112304+0000", 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112310+0000", 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1859999999999997e-06 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:23.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112315+0000", 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119189+0000", 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068746950000000001 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119192+0000", 2024-09-13T09:09:23.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131128+0000", 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01193613 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131134+0000", 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0240000000000003e-06 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154251 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8223248229999998, 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018254725, 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:23.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154251 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:23.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112899+0000", 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112905+0000", 2024-09-13T09:09:24.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5709999999999997e-06 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112916+0000", 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112923+0000", 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119201+0000", 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062780379999999997 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119204+0000", 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131151+0000", 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011947269 2024-09-13T09:09:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131155+0000", 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.284e-06 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154252 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:24.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8211833989999997, 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171325529999999, 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154252 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114041+0000", 2024-09-13T09:09:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114046+0000", 2024-09-13T09:09:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T09:09:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114062+0000", 2024-09-13T09:09:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5951999999999999e-05 2024-09-13T09:09:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114070+0000", 2024-09-13T09:09:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9510000000000006e-06 2024-09-13T09:09:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119213+0000", 2024-09-13T09:09:24.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051428860000000002 2024-09-13T09:09:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119215+0000", 2024-09-13T09:09:24.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4509999999999999e-06 2024-09-13T09:09:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131171+0000", 2024-09-13T09:09:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119552249999999 2024-09-13T09:09:24.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131174+0000", 2024-09-13T09:09:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6739999999999999e-06 2024-09-13T09:09:24.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154253 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8211481889999996, 2024-09-13T09:09:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017116229, 2024-09-13T09:09:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:24.014 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154253 2024-09-13T09:09:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.015 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114080+0000", 2024-09-13T09:09:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114087+0000", 2024-09-13T09:09:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1679999999999998e-06 2024-09-13T09:09:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114092+0000", 2024-09-13T09:09:24.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119225+0000", 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051325920000000001 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119227+0000", 2024-09-13T09:09:24.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131187+0000", 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011959702 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131193+0000", 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154260 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8211114300000002, 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017101802, 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154260 2024-09-13T09:09:24.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114113+0000", 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114117+0000", 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3139999999999999e-06 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114125+0000", 2024-09-13T09:09:24.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1429999999999998e-06 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114131+0000", 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119237+0000", 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051056859999999999 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119239+0000", 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131212+0000", 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119728100000001 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131216+0000", 2024-09-13T09:09:24.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7160000000000001e-06 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154271 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8210889750000003, 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017099328, 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154271 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114138+0000", 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114145+0000", 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114150+0000", 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119251+0000", 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051011809999999998 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119256+0000", 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131232+0000", 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011976298 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131236+0000", 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6030000000000001e-06 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154272 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8210712959999995, 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017104572, 2024-09-13T09:09:24.033 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154272 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114156+0000", 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114162+0000", 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114167+0000", 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.977e-06 2024-09-13T09:09:24.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119274+0000", 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051065360000000001 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119279+0000", 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131252+0000", 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011972874 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131259+0000", 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154273 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8210534520000001, 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171116979999999, 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154273 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114171+0000", 2024-09-13T09:09:24.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114174+0000", 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114181+0000", 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5030000000000002e-06 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114185+0000", 2024-09-13T09:09:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6720000000000003e-06 2024-09-13T09:09:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119297+0000", 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051116549999999997 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119302+0000", 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131276+0000", 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011973451 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131283+0000", 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.627e-06 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154274 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8210371519999997, 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171203449999999, 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154274 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114190+0000", 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7519999999999999e-06 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114193+0000", 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114195+0000", 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.407e-06 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119320+0000", 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051245739999999998 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119323+0000", 2024-09-13T09:09:24.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131301+0000", 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01197782 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131308+0000", 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5440000000000001e-06 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154293 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8180088489999999, 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0141231429999999, 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154293 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117215+0000", 2024-09-13T09:09:24.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117223+0000", 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117236+0000", 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3492000000000001e-05 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117247+0000", 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0725e-05 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119334+0000", 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002087392 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119337+0000", 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:09:24.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131332+0000", 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119952649999999 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131340+0000", 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2089999999999999e-06 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.056 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.056 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:24.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:24.058 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:24.054+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:24.158 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:24.154+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:24.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:09:24.221 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1] out_osds: [7, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:24.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:09:24.242 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:24.242 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:24.242 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:24.242 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:24.243 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:24.243 INFO:teuthology.orchestra.run.smithi031.stdout:5900 read 0x74d7711 thru 0x74e2026 (0xa916 bytes) 2024-09-13T09:09:24.244 INFO:teuthology.orchestra.run.smithi031.stdout:5901 write 0x721b683 thru 0x7221e61 (0x67df bytes) 2024-09-13T09:09:24.258 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:24.254+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:24.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:24.263 INFO:teuthology.orchestra.run.smithi031.stdout:5902 punch from 0xde8a73e to 0xde8d5ee, (0x2eb0 bytes) 2024-09-13T09:09:24.284 INFO:teuthology.orchestra.run.smithi031.stdout:5905 trunc from 0xee18bcc to 0x1c29cd2 2024-09-13T09:09:24.340 INFO:teuthology.orchestra.run.smithi031.stdout:5909 punch from 0x1c272dc to 0x1c29cd2, (0x29f6 bytes) 2024-09-13T09:09:24.356 INFO:teuthology.orchestra.run.smithi031.stdout:5910 write 0x72e451e thru 0x72f35b7 (0xf09a bytes) 2024-09-13T09:09:24.359 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:24.358+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:24.380 INFO:teuthology.orchestra.run.smithi031.stdout:5911 write 0x850bfc1 thru 0x850e160 (0x21a0 bytes) 2024-09-13T09:09:24.389 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:24.392 INFO:teuthology.orchestra.run.smithi031.stdout:5914 trunc from 0x850e161 to 0x673108c 2024-09-13T09:09:24.401 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:24.401 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:24.413 INFO:teuthology.orchestra.run.smithi031.stdout:5915 write 0x3482f2a thru 0x3489505 (0x65dc bytes) 2024-09-13T09:09:24.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:24.425 INFO:teuthology.orchestra.run.smithi031.stdout:5917 trunc from 0x673108c to 0x6db4786 2024-09-13T09:09:24.433 INFO:teuthology.orchestra.run.smithi031.stdout:5918 read 0x20ea27f thru 0x20f53a3 (0xb125 bytes) 2024-09-13T09:09:24.433 INFO:teuthology.orchestra.run.smithi031.stdout:5919 read 0x258ea19 thru 0x2592236 (0x381e bytes) 2024-09-13T09:09:24.452 INFO:teuthology.orchestra.run.smithi031.stdout:5920 write 0xdbc4667 thru 0xdbd0762 (0xc0fc bytes) 2024-09-13T09:09:24.460 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:24.458+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:24.466 INFO:teuthology.orchestra.run.smithi031.stdout:5921 writesame 0x8f0bd6 thru 0x8fbdf7 data_size 0x2(0xb222 bytes) 2024-09-13T09:09:24.496 INFO:teuthology.orchestra.run.smithi031.stdout:5922 write 0x3de8212 thru 0x3de967d (0x146c bytes) 2024-09-13T09:09:24.504 INFO:teuthology.orchestra.run.smithi031.stdout:5923 punch from 0x8e3be54 to 0x8e3e448, (0x25f4 bytes) 2024-09-13T09:09:24.504 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:24.504 INFO:teuthology.orchestra.run.smithi031.stdout:5925 clone 47 order 18 su 8192 sc 12 2024-09-13T09:09:24.552 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212463605, 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:24.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:09:24.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212446832, 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:09:24.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:09:24.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21240952700001, 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:09:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:09:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212389813, 2024-09-13T09:09:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:09:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:09:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:09:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212373488, 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:09:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:09:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:09:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212357622, 2024-09-13T09:09:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:09:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:09:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:09:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212341837, 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:09:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:09:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212322766, 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:09:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212306219, 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:09:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21227862000001, 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21226137799999, 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:09:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21223093499999, 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:09:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:09:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:09:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21221202, 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212194232, 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:09:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:09:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212169008, 2024-09-13T09:09:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:09:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.212150804, 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:09:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21213048600001, 2024-09-13T09:09:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:09:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:09:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:09:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21211193800001, 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:09:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:09:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:09:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:09:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21209466400001, 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.21207721, 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:09:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:24.654 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:24.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:24.655 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:24.648+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:24.755 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:24.754+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:24.832 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:24.833 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:24.833 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:24.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:24.888 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:24.886+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:24.958 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:24.956+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:25.012 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:25.012 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:25.012 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:25.012 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:25.012 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:25.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:25.058 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:25.056+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 141.02218119, 2024-09-13T09:09:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:09:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:09:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:09:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.193899009999996, 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.901459172000003, 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:09:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.901379374000001, 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:09:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:09:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.901130303000002, 2024-09-13T09:09:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:09:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:09:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:09:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:09:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.900817161999996, 2024-09-13T09:09:25.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:09:25.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:09:25.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:09:25.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.900363929999997, 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:09:25.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.900339415999994, 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.900074187000001, 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:09:25.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.899786337999998, 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:25.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 84.899325034, 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:09:25.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:09:25.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:09:25.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:09:25.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.960791684999997, 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:25.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.949798364999999, 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:09:25.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:09:25.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:09:25.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.945816467, 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:09:25.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:09:25.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:25.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.943188472999999, 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:09:25.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:25.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:09:25.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.941635689000002, 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:09:25.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:09:25.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:09:25.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:09:25.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.252 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.940382372999998, 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:09:25.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:09:25.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:09:25.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:09:25.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.939280340000003, 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:09:25.260 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:09:25.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:09:25.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.938069763000001, 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:09:25.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:09:25.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:09:25.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:09:25.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3:f1cafc87:::rbd_data.1124651f5f97.000000000000004a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 557056~22319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.030374348, 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97276468599999999, 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:25.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092738+0000", 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97003675 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092877+0000", 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139245 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093029+0000", 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015224999999999999 2024-09-13T09:09:25.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093847+0000", 2024-09-13T09:09:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081788800000000001 2024-09-13T09:09:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:25.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095250+0000", 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014028529999999999 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095282+0000", 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1961000000000001e-05 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095303+0000", 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:25.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:25.282 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:25.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:09:25.284 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:09:25.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:09:25.286 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone46 intact 2024-09-13T09:09:25.287 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:25.276+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:25.384 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:25.376+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:25.420 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:25.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:25.420 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:25.420 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:25.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:25.472 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-13T09:09:25.525 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:25.520+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:25.585 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:25.580+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:25.623 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:25.623 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:25.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:25.687 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:25.680+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:25.788 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:25.780+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786965187000007, 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:09:25.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:09:25.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786917853999995, 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:09:25.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:25.824 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786860317999995, 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:25.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:09:25.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:09:25.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:09:25.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:09:25.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:25.829 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786808832000006, 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:25.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:09:25.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:09:25.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786761889999994, 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.834 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:09:25.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.838 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786714497000006, 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:09:25.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:09:25.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:09:25.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:09:25.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786667222000005, 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:25.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786621022000006, 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:09:25.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.786557017000007, 2024-09-13T09:09:25.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:25.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.784085927999996, 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:09:25.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.784030205999997, 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:09:25.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.868 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.783980303000007, 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.869 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:09:25.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:25.873 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.783933246999993, 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:09:25.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:09:25.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.783884764000007, 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:09:25.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.783819499000003, 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.884 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:25.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:25.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:09:25.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.783772583000001, 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.891 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:09:25.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.783685134999999, 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.896 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:09:25.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.900 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.782223286000004, 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.901 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:25.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:09:25.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.782145200000002, 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:09:25.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:25.910 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 77.782093110999995, 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:09:25.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:25.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:09:25.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:09:25.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.915 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.915 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:25.915 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:25.915 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:25.915 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:25.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:25.918 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:25.912+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:25.971 INFO:teuthology.orchestra.run.smithi031.stdout:5927 punch from 0xd1e6f9c to 0xd1ec001, (0x5065 bytes) 2024-09-13T09:09:25.977 INFO:teuthology.orchestra.run.smithi031.stdout:5928 punch from 0xc19695 to 0xc26215, (0xcb80 bytes) 2024-09-13T09:09:25.977 INFO:teuthology.orchestra.run.smithi031.stdout:5929 read 0x3759e71 thru 0x375e8b1 (0x4a41 bytes) 2024-09-13T09:09:25.978 INFO:teuthology.orchestra.run.smithi031.stdout:5931 write 0x284ca5e thru 0x284f1f1 (0x2794 bytes) 2024-09-13T09:09:25.984 INFO:teuthology.orchestra.run.smithi031.stdout:5932 punch from 0x8d79775 to 0x8d81799, (0x8024 bytes) 2024-09-13T09:09:25.984 INFO:teuthology.orchestra.run.smithi031.stdout:5933 punch from 0xa8ff31 to 0xa9fdbf, (0xfe8e bytes) 2024-09-13T09:09:25.984 INFO:teuthology.orchestra.run.smithi031.stdout:5935 punch from 0x64ee8d5 to 0x64f7b11, (0x923c bytes) 2024-09-13T09:09:25.984 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:25.985 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:26.002 INFO:teuthology.orchestra.run.smithi031.stdout:5938 trunc from 0xdbd0763 to 0xbc8501 2024-09-13T09:09:26.018 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:26.014+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:26.062 INFO:teuthology.orchestra.run.smithi031.stdout:5939 write 0x84317df thru 0x843a5ef (0x8e11 bytes) 2024-09-13T09:09:26.072 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:26.072 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:26.075 INFO:teuthology.orchestra.run.smithi031.stdout:5940 writesame 0x81fb5ec thru 0x82096e8 data_size 0x49(0xe0fd bytes) 2024-09-13T09:09:26.092 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:26.092 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:26.093 INFO:teuthology.orchestra.run.smithi031.stdout:5941 writesame 0x5160a2c thru 0x5162bad data_size 0x1(0x2182 bytes) 2024-09-13T09:09:26.102 INFO:teuthology.orchestra.run.smithi031.stdout:5942 writesame 0x412e8d8 thru 0x412fa00 data_size 0x1(0x1129 bytes) 2024-09-13T09:09:26.108 INFO:teuthology.orchestra.run.smithi031.stdout:5943 read 0x7fa523 thru 0x805ced (0xb7cb bytes) 2024-09-13T09:09:26.108 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:26.109 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:26.109 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:26.109 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:26.110 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:26.110 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:26.110 INFO:teuthology.orchestra.run.smithi031.stdout:5944 read 0x27cf246 thru 0x27db145 (0xbf00 bytes) 2024-09-13T09:09:26.110 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:26.110 INFO:teuthology.orchestra.run.smithi031.stdout:5947 read 0x633bc7e thru 0x633d6e7 (0x1a6a bytes) 2024-09-13T09:09:26.110 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:26.117 INFO:teuthology.orchestra.run.smithi031.stdout:5950 trunc from 0x843a5f0 to 0xd823c7b 2024-09-13T09:09:26.119 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:26.114+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:26.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:26.126 INFO:teuthology.orchestra.run.smithi031.stdout:5951 write 0x259c0bd thru 0x25a4983 (0x88c7 bytes) 2024-09-13T09:09:26.127 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:26.128 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:26.135 INFO:teuthology.orchestra.run.smithi031.stdout:5953 writesame 0x144363e thru 0x1447991 data_size 0x3e(0x4354 bytes) 2024-09-13T09:09:26.142 INFO:teuthology.orchestra.run.smithi031.stdout:5955 punch from 0x8456ba5 to 0x845fc2e, (0x9089 bytes) 2024-09-13T09:09:26.150 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:26.150 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:26.150 INFO:teuthology.orchestra.run.smithi031.stdout:5956 write 0xe2b539b thru 0xe2ba57b (0x51e1 bytes) 2024-09-13T09:09:26.158 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:26.159 INFO:teuthology.orchestra.run.smithi031.stdout:5961 trunc from 0xe2ba57c to 0xdf1b034 2024-09-13T09:09:26.169 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:26.170 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:26.171 INFO:teuthology.orchestra.run.smithi031.stdout:5962 write 0xd2a5a0b thru 0xd2aee4e (0x9444 bytes) 2024-09-13T09:09:26.175 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:09:26.185 INFO:teuthology.orchestra.run.smithi031.stdout:5965 clone 48 order 22 su 1048576 sc 13 2024-09-13T09:09:26.193 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:26.193 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:26.200 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:09:26.213 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:26.214 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:26.220 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:26.216+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:26.251 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:26.251 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:26.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:26.357 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:26.352+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:26.422 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:26.416+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:26.439 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:26.439 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:26.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:26.522 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:26.518+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:26.622 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:26.622 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:26.623 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:26.616+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:26.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:26.725 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:26.720+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:26.808 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:26.808 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:26.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:26.865 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:26.860+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:26.926 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:26.922+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:26.984 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:26.985 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:26.999 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:27.045 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:27.040+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:27.125 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:27.125 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:27.128 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:27.120+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:27.139 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:27.194 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:09:27.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:09:27.233 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:27.228+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:27.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:09:27.319 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone47 from 0xdf1b034 (overlap 0xbc8501) to 0x4b6866 2024-09-13T09:09:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:27.321 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:27.329 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:27.326+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:27.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:27.430 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:27.426+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:27.487 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-13T09:09:27.506 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:27.507 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:27.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:27.531 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:27.524+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:27.632 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:27.630+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:27.666 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:27.666 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:27.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:27.733 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:27.728+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:27.834 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:27.828+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:27.854 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:27.854 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:27.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:27.935 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:27.928+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:27.963 INFO:teuthology.orchestra.run.smithi031.stdout:5966 read 0x3a5bba7 thru 0x3a62e97 (0x72f1 bytes) 2024-09-13T09:09:27.971 INFO:teuthology.orchestra.run.smithi031.stdout:5969 trunc from 0xdf1b034 to 0x8d1dc8c 2024-09-13T09:09:28.003 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:28.003 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:28.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:28.031 INFO:teuthology.orchestra.run.smithi031.stdout:5970 punch from 0x52ed44e to 0x52f2319, (0x4ecb bytes) 2024-09-13T09:09:28.031 INFO:teuthology.orchestra.run.smithi031.stdout:5971 flatten 2024-09-13T09:09:28.037 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:28.032+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:28.097 INFO:teuthology.orchestra.run.smithi031.stdout:5973 read 0x7109b73 thru 0x7115870 (0xbcfe bytes) 2024-09-13T09:09:28.097 INFO:teuthology.orchestra.run.smithi031.stdout:5975 write 0x7029bbf thru 0x703064a (0x6a8c bytes) 2024-09-13T09:09:28.109 INFO:teuthology.orchestra.run.smithi031.stdout:5976 write 0xa68e8ab thru 0xa6982e2 (0x9a38 bytes) 2024-09-13T09:09:28.114 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:28.114 INFO:teuthology.orchestra.run.smithi031.stdout:5978 write 0x931c136 thru 0x9320cf0 (0x4bbb bytes) 2024-09-13T09:09:28.117 INFO:teuthology.orchestra.run.smithi031.stdout:5979 punch from 0x8af715a to 0x8afb147, (0x3fed bytes) 2024-09-13T09:09:28.127 INFO:teuthology.orchestra.run.smithi031.stdout:5980 trunc from 0xa6982e3 to 0x2114d4b 2024-09-13T09:09:28.137 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:28.134+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:28.153 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:28.153 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:28.155 INFO:teuthology.orchestra.run.smithi031.stdout:5981 read 0x1851348 thru 0x18529e4 (0x169d bytes) 2024-09-13T09:09:28.156 INFO:teuthology.orchestra.run.smithi031.stdout:5983 read 0x134b393 thru 0x134e78c (0x33fa bytes) 2024-09-13T09:09:28.169 INFO:teuthology.orchestra.run.smithi031.stdout:5986 trunc from 0x2114d4b to 0x7e276ec 2024-09-13T09:09:28.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:5988 punch from 0xb907e3 to 0xb98426, (0x7c43 bytes) 2024-09-13T09:09:28.178 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-09-13T09:09:28.179 INFO:teuthology.orchestra.run.smithi031.stdout:5993 writesame 0x423f65a thru 0x424403e data_size 0x1(0x49e5 bytes) 2024-09-13T09:09:28.194 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:09:28.195 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:09:28.216 INFO:teuthology.orchestra.run.smithi031.stdout:5995 write 0xee1b084 thru 0xee1e0a2 (0x301f bytes) 2024-09-13T09:09:28.224 INFO:teuthology.orchestra.run.smithi031.stdout:5996 read 0xe4a6268 thru 0xe4b6177 (0xff10 bytes) 2024-09-13T09:09:28.226 INFO:teuthology.orchestra.run.smithi031.stdout:5997 trunc from 0xee1e0a3 to 0xd8adeb9 2024-09-13T09:09:28.229 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:09:28.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:28.238 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:28.232+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:28.246 INFO:teuthology.orchestra.run.smithi031.stdout:5999 write 0xdbd69ac thru 0xdbdb846 (0x4e9b bytes) 2024-09-13T09:09:28.263 INFO:teuthology.orchestra.run.smithi031.stdout:6000 trunc from 0xdbdb847 to 0x4647377 2024-09-13T09:09:28.314 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:28.314 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:28.329 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-13T09:09:28.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:28.375 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:28.374+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:28.397 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:28.416 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:28.416 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:28.435 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:28.435 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:28.440 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:28.438+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:28.453 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:28.454 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:28.471 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154208 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3583690359999991, 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019831428, 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.472 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154208 2024-09-13T09:09:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110898+0000", 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110909+0000", 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0896999999999999e-05 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110932+0000", 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3544999999999999e-05 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110943+0000", 2024-09-13T09:09:28.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.064e-05 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119027+0000", 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080840670000000003 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119033+0000", 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0050000000000001e-06 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130694+0000", 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116608949999999 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130732+0000", 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7696e-05 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154209 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3583109790000005, 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199906329999999, 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154209 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:09:28.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110961+0000", 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110967+0000", 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4470000000000003e-06 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110972+0000", 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119044+0000", 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080713150000000008 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119046+0000", 2024-09-13T09:09:28.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130938+0000", 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0118917460000001 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130949+0000", 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0607e-05 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154218 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3582350989999998, 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019944518, 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154218 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111033+0000", 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111038+0000", 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111047+0000", 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1779999999999992e-06 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111053+0000", 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9089999999999997e-06 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119060+0000", 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080075760000000006 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119065+0000", 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130971+0000", 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119057490000001 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130979+0000", 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8790000000000003e-06 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154219 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3581933189999997, 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199227849999999, 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154219 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111075+0000", 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111079+0000", 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111086+0000", 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111090+0000", 2024-09-13T09:09:28.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119079+0000", 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007988396 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119081+0000", 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130995+0000", 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011913643 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130999+0000", 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6940000000000001e-06 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154228 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.492 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3578529629999991, 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019595799, 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154228 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111415+0000", 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111421+0000", 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111432+0000", 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1807000000000001e-05 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111439+0000", 2024-09-13T09:09:28.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8909999999999998e-06 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119092+0000", 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076528159999999998 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119097+0000", 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131008+0000", 2024-09-13T09:09:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011911539 2024-09-13T09:09:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131012+0000", 2024-09-13T09:09:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5760000000000002e-06 2024-09-13T09:09:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.497 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154229 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3570873480000003, 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01884818, 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.498 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154229 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112181+0000", 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112186+0000", 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249999999999997e-06 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112202+0000", 2024-09-13T09:09:28.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.543e-05 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112209+0000", 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5669999999999997e-06 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119115+0000", 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069053999999999999 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119120+0000", 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131023+0000", 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011903446 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131030+0000", 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154238 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.357053037, 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188343550000001, 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154238 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112219+0000", 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0240000000000002e-06 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112225+0000", 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112230+0000", 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119130+0000", 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068996450000000003 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119132+0000", 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131046+0000", 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119135260000001 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131050+0000", 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5859999999999998e-06 2024-09-13T09:09:28.507 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154239 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3570345629999991, 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188390510000001, 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154239 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:09:28.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112234+0000", 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112237+0000", 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112243+0000", 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112247+0000", 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119149+0000", 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069020679999999999 2024-09-13T09:09:28.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119154+0000", 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131067+0000", 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011912659 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131074+0000", 2024-09-13T09:09:28.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9709999999999998e-06 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154242 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3570151609999996, 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188413620000001, 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154242 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112256+0000", 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112262+0000", 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-13T09:09:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112267+0000", 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119166+0000", 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068991599999999997 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119168+0000", 2024-09-13T09:09:28.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.678e-06 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131088+0000", 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011919435 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131095+0000", 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5769999999999996e-06 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154243 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3569873090000009, 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018836657, 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154243 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.518 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112281+0000", 2024-09-13T09:09:28.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112285+0000", 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112292+0000", 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.126e-06 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112297+0000", 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0760000000000002e-06 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119178+0000", 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068802890000000004 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119180+0000", 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T09:09:28.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131111+0000", 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011930947 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131119+0000", 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154244 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.356967161, 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188321789999999, 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154244 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112304+0000", 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112310+0000", 2024-09-13T09:09:28.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1859999999999997e-06 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112315+0000", 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119189+0000", 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068746950000000001 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119192+0000", 2024-09-13T09:09:28.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131128+0000", 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01193613 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131134+0000", 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0240000000000003e-06 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154251 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3563687709999996, 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018254725, 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154251 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112899+0000", 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:28.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112905+0000", 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5709999999999997e-06 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112916+0000", 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112923+0000", 2024-09-13T09:09:28.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119201+0000", 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062780379999999997 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119204+0000", 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131151+0000", 2024-09-13T09:09:28.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011947269 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131155+0000", 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.284e-06 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154252 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:28.532 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3552273469999996, 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171325529999999, 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154252 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114041+0000", 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114046+0000", 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T09:09:28.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114062+0000", 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5951999999999999e-05 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114070+0000", 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9510000000000006e-06 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119213+0000", 2024-09-13T09:09:28.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051428860000000002 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119215+0000", 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4509999999999999e-06 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131171+0000", 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119552249999999 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131174+0000", 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6739999999999999e-06 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154253 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3551921369999995, 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017116229, 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.537 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154253 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:28.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114080+0000", 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114087+0000", 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1679999999999998e-06 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114092+0000", 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119225+0000", 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051325920000000001 2024-09-13T09:09:28.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119227+0000", 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131187+0000", 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011959702 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131193+0000", 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154260 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3551553779999992, 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017101802, 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154260 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114113+0000", 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114117+0000", 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3139999999999999e-06 2024-09-13T09:09:28.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114125+0000", 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1429999999999998e-06 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114131+0000", 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119237+0000", 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051056859999999999 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119239+0000", 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131212+0000", 2024-09-13T09:09:28.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119728100000001 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131216+0000", 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7160000000000001e-06 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154271 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:28.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3551329229999993, 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017099328, 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154271 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:28.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114138+0000", 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114145+0000", 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114150+0000", 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119251+0000", 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051011809999999998 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119256+0000", 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131232+0000", 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011976298 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131236+0000", 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6030000000000001e-06 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154272 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3551152440000003, 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017104572, 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154272 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114156+0000", 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-13T09:09:28.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114162+0000", 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114167+0000", 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.977e-06 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119274+0000", 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051065360000000001 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119279+0000", 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131252+0000", 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011972874 2024-09-13T09:09:28.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131259+0000", 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154273 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3550974, 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171116979999999, 2024-09-13T09:09:28.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154273 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114171+0000", 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114174+0000", 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:09:28.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114181+0000", 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5030000000000002e-06 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114185+0000", 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6720000000000003e-06 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119297+0000", 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051116549999999997 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119302+0000", 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131276+0000", 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011973451 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131283+0000", 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.627e-06 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154274 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3550810999999996, 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171203449999999, 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154274 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114190+0000", 2024-09-13T09:09:28.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7519999999999999e-06 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114193+0000", 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114195+0000", 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.407e-06 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119320+0000", 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051245739999999998 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119323+0000", 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131301+0000", 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01197782 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131308+0000", 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5440000000000001e-06 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154293 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3520527970000007, 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0141231429999999, 2024-09-13T09:09:28.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154293 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117215+0000", 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117223+0000", 2024-09-13T09:09:28.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117236+0000", 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3492000000000001e-05 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117247+0000", 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0725e-05 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119334+0000", 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002087392 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119337+0000", 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131332+0000", 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119952649999999 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:28.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131340+0000", 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2089999999999999e-06 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:28.573 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:28.574 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:28.575 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:28.576 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:28.576 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:28.576 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:28.577 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:28.577 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:28.578 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:28.578 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:28.568+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:28.675 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:28.674+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:28.729 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:28.729 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:28.729 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:28.729 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:28.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:28.788 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:28.786+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:28.877 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:28.876+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:28.907 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:28.907 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:28.907 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:28.907 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:28.907 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:28.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:28.978 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:28.974+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:29.043 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:29.043 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:29.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.70200611, 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:09:29.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:09:29.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:09:29.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:09:29.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:09:29.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701989337, 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:09:29.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:09:29.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:09:29.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:09:29.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.70195203199999, 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701932318, 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:09:29.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:09:29.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.063 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701915993, 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:29.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:09:29.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701900127, 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:09:29.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:09:29.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701884342, 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:09:29.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701865271, 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:29.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:09:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:09:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:09:29.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701848724, 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.083 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:09:29.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701821125, 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701803883, 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:09:29.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.70177344, 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:09:29.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701754525, 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:09:29.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:09:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701736737, 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.70171151300001, 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:29.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.70169330900001, 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:09:29.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701672991, 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.701654443, 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.127 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:09:29.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.131 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.70163716899999, 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:09:29.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:29.136 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.70161971500001, 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:09:29.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.141 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:29.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:29.143 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:29.138+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:29.204 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:09:29.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:09:29.279 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:29.278+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:29.326 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:29.326 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:29.326 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:29.343 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:29.392 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:29.390+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:29.446 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:29.440+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:29.512 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:29.513 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:29.513 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:29.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:29.546 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:29.540+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:29.647 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:29.646+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:29.650 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:29.650 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:29.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:29.650 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:29.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 145.51569211899999, 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:09:29.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:09:29.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:09:29.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 96.687409939000005, 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:09:29.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:09:29.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.394970100999998, 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:09:29.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.667 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.394890302999997, 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:29.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:09:29.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.394641231999998, 2024-09-13T09:09:29.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:09:29.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.394328091000006, 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:09:29.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.393874858999993, 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:09:29.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.393850345000004, 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:09:29.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:09:29.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:09:29.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.393585115999997, 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:09:29.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.393297266999994, 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:09:29.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:09:29.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.392835962999996, 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:09:29.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:09:29.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:29.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.454302613999999, 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:09:29.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:09:29.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:09:29.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.443309294000002, 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:09:29.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:09:29.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:09:29.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.439327396000003, 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:29.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:09:29.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:09:29.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.436699402000002, 2024-09-13T09:09:29.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:09:29.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:09:29.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.435146617999997, 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:09:29.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:09:29.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:09:29.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:09:29.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.433893302000001, 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:09:29.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:09:29.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.432791268999999, 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.745 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:09:29.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:09:29.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:09:29.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:29.751 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.431580691999997, 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:09:29.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:09:29.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3:f1cafc87:::rbd_data.1124651f5f97.000000000000004a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 557056~22319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.523885277, 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97276468599999999, 2024-09-13T09:09:29.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:09:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092738+0000", 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97003675 2024-09-13T09:09:29.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092877+0000", 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139245 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093029+0000", 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015224999999999999 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:29.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093847+0000", 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081788800000000001 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095250+0000", 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014028529999999999 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095282+0000", 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1961000000000001e-05 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:29.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095303+0000", 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:29.765 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:29.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:29.767 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:29.762+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:29.868 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:29.866+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:29.902 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:29.903 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:29.903 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:29.903 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:29.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:29.969 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:29.964+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:30.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:30.097 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:30.092+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:30.170 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:30.164+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:30.215 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:09:30.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:09:30.232 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:30.232 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:30.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:30.232 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:30.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.204162597999996, 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.233 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:09:30.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:30.237 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.204115264999999, 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:09:30.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.204057728999999, 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:09:30.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.246 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.204006242999995, 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:09:30.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.203959300999998, 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.251 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:30.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:09:30.255 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.203911907999995, 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.256 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:09:30.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.260 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.203864632999995, 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:09:30.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.265 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.203818432999995, 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:09:30.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.203754427999996, 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:09:30.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.274 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.201283339, 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.275 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.201227617000001, 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.280 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:09:30.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.284 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.201177713999996, 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:09:30.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.201130657999997, 2024-09-13T09:09:30.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:09:30.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.201082174999996, 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.294 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:09:30.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.201016910000007, 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:09:30.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:30.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:09:30.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.200969994000005, 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:09:30.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:09:30.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:30.308 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.200882546000003, 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:09:30.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.199420696999994, 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:09:30.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.317 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.199342611000006, 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:09:30.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:09:30.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:09:30.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:30.322 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 82.199290521999998, 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:30.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:09:30.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:30.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:09:30.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:09:30.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.327 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:30.327 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:30.327 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:30.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:30.330 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:30.324+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:30.330 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:09:30.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:09:30.430 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:30.424+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:30.508 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:30.508 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:30.508 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:30.508 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:30.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:30.531 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:30.526+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:30.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:30.632 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:30.630+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:30.673 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:09:30.674 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:09:30.689 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:30.690 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:30.692 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-13T09:09:30.210833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1559401649,"num_objects":1806,"num_object_clones":390,"num_object_copies":3612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1806,"num_whiteouts":662,"num_read":196104,"num_read_kb":664646,"num_write":22470,"num_write_kb":329469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":175529217,"num_keys_recovered":405,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17728,"ondisk_log_size":17728,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":661124,"kb_used_data":359620,"kb_used_omap":9,"kb_used_meta":301430,"kb_avail":565569916,"statfs":{"total":579820584960,"available":579143593984,"internally_reserved":0,"allocated":368250880,"data_stored":398563775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":308664999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10589861,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":7,"num_read":2942,"num_read_kb":13202,"num_write":379,"num_write_kb":2852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.038058"},"pg_stats":[{"pgid":"3.1f","version":"130'494","reported_seq":5374,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:28.146211+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:09:28.146211+0000","last_peered":"2024-09-13T09:09:28.146211+0000","last_clean":"2024-09-13T09:09:28.146211+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:09:28.146211+0000","last_undegraded":"2024-09-13T09:09:28.146211+0000","last_fullsized":"2024-09-13T09:09:28.146211+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":40723036,"num_objects":65,"num_object_clones":16,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":28,"num_read":5902,"num_read_kb":20167,"num_write":631,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'1060","reported_seq":4403,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:28.135341+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:09:28.135341+0000","last_peered":"2024-09-13T09:09:28.135341+0000","last_clean":"2024-09-13T09:09:28.135341+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:09:28.135341+0000","last_undegraded":"2024-09-13T09:09:28.135341+0000","last_fullsized":"2024-09-13T09:09:28.135341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1060,"log_dups_size":0,"ondisk_log_size":1060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":48051095,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":26,"num_read":8453,"num_read_kb":18493,"num_write":1447,"num_write_kb":5996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"127'1332","reported_seq":5059,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175441+0000","last_change":"2024-09-13T09:08:38.210944+0000","last_active":"2024-09-13T09:09:25.175441+0000","last_peered":"2024-09-13T09:09:25.175441+0000","last_clean":"2024-09-13T09:09:25.175441+0000","last_became_active":"2024-09-13T09:08:38.208191+0000","last_became_peered":"2024-09-13T09:08:38.208191+0000","last_unstale":"2024-09-13T09:09:25.175441+0000","last_undegraded":"2024-09-13T09:09:25.175441+0000","last_fullsized":"2024-09-13T09:09:25.175441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1332,"log_dups_size":0,"ondisk_log_size":1332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:04.686300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":54547238,"num_objects":68,"num_object_clones":17,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":8690,"num_read_kb":33724,"num_write":2192,"num_write_kb":17061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"129'572","reported_seq":3382,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.184373+0000","last_change":"2024-09-13T09:08:38.212078+0000","last_active":"2024-09-13T09:09:26.184373+0000","last_peered":"2024-09-13T09:09:26.184373+0000","last_clean":"2024-09-13T09:09:26.184373+0000","last_became_active":"2024-09-13T09:08:38.209451+0000","last_became_peered":"2024-09-13T09:08:38.209451+0000","last_unstale":"2024-09-13T09:09:26.184373+0000","last_undegraded":"2024-09-13T09:09:26.184373+0000","last_fullsized":"2024-09-13T09:09:26.184373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:59.672566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":79776230,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":19,"num_read":5156,"num_read_kb":34819,"num_write":801,"num_write_kb":15621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"127'577","reported_seq":21506,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.399055+0000","last_change":"2024-09-13T09:08:38.212581+0000","last_active":"2024-09-13T09:09:24.399055+0000","last_peered":"2024-09-13T09:09:24.399055+0000","last_clean":"2024-09-13T09:09:24.399055+0000","last_became_active":"2024-09-13T09:08:38.210333+0000","last_became_peered":"2024-09-13T09:08:38.210333+0000","last_unstale":"2024-09-13T09:09:24.399055+0000","last_undegraded":"2024-09-13T09:09:24.399055+0000","last_fullsized":"2024-09-13T09:09:24.399055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:53.493070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":94460825,"num_objects":61,"num_object_clones":12,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":19,"num_read":22971,"num_read_kb":47577,"num_write":676,"num_write_kb":18535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"127'574","reported_seq":5403,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.170959+0000","last_change":"2024-09-13T09:08:09.864519+0000","last_active":"2024-09-13T09:09:26.170959+0000","last_peered":"2024-09-13T09:09:26.170959+0000","last_clean":"2024-09-13T09:09:26.170959+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:09:26.170959+0000","last_undegraded":"2024-09-13T09:09:26.170959+0000","last_fullsized":"2024-09-13T09:09:26.170959+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":62903922,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":26,"num_read":7305,"num_read_kb":23969,"num_write":758,"num_write_kb":19650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"125'323","reported_seq":1676,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.474591+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:09:24.474591+0000","last_peered":"2024-09-13T09:09:24.474591+0000","last_clean":"2024-09-13T09:09:24.474591+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:09:24.474591+0000","last_undegraded":"2024-09-13T09:09:24.474591+0000","last_fullsized":"2024-09-13T09:09:24.474591+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":15863077,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":1622,"num_read_kb":7909,"num_write":307,"num_write_kb":7298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"130'579","reported_seq":2916,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:28.134903+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:09:28.134903+0000","last_peered":"2024-09-13T09:09:28.134903+0000","last_clean":"2024-09-13T09:09:28.134903+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:09:28.134903+0000","last_undegraded":"2024-09-13T09:09:28.134903+0000","last_fullsized":"2024-09-13T09:09:28.134903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":47319500,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":26,"num_read":4163,"num_read_kb":14101,"num_write":675,"num_write_kb":10427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'532","reported_seq":5194,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T09:09:28.195785+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:09:28.195785+0000","last_peered":"2024-09-13T09:09:28.195785+0000","last_clean":"2024-09-13T09:09:28.195785+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:09:28.195785+0000","last_undegraded":"2024-09-13T09:09:28.195785+0000","last_fullsized":"2024-09-13T09:09:28.195785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":55833489,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":17,"num_read":6428,"num_read_kb":30595,"num_write":549,"num_write_kb":18247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"128'641","reported_seq":1705,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.135087+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:09:26.135087+0000","last_peered":"2024-09-13T09:09:26.135087+0000","last_clean":"2024-09-13T09:09:26.135087+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:09:26.135087+0000","last_undegraded":"2024-09-13T09:09:26.135087+0000","last_fullsized":"2024-09-13T09:09:26.135087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":36375948,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":2806,"num_read_kb":13605,"num_write":920,"num_write_kb":9801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"128'569","reported_seq":3713,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.211835+0000","last_change":"2024-09-13T09:08:38.222834+0000","last_active":"2024-09-13T09:09:25.211835+0000","last_peered":"2024-09-13T09:09:25.211835+0000","last_clean":"2024-09-13T09:09:25.211835+0000","last_became_active":"2024-09-13T09:08:38.221484+0000","last_became_peered":"2024-09-13T09:08:38.221484+0000","last_unstale":"2024-09-13T09:09:25.211835+0000","last_undegraded":"2024-09-13T09:09:25.211835+0000","last_fullsized":"2024-09-13T09:09:25.211835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:04.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":93809147,"num_objects":66,"num_object_clones":20,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":21,"num_read":4632,"num_read_kb":23977,"num_write":589,"num_write_kb":14829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"128'582","reported_seq":18691,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.185120+0000","last_change":"2024-09-13T09:09:24.174071+0000","last_active":"2024-09-13T09:09:26.185120+0000","last_peered":"2024-09-13T09:09:26.185120+0000","last_clean":"2024-09-13T09:09:26.185120+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:09:26.185120+0000","last_undegraded":"2024-09-13T09:09:26.185120+0000","last_fullsized":"2024-09-13T09:09:26.185120+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":47148079,"num_objects":60,"num_object_clones":15,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":18,"num_read":20373,"num_read_kb":32741,"num_write":542,"num_write_kb":9900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175644+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:09:25.175644+0000","last_peered":"2024-09-13T09:09:25.175644+0000","last_clean":"2024-09-13T09:09:25.175644+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:09:25.175644+0000","last_undegraded":"2024-09-13T09:09:25.175644+0000","last_fullsized":"2024-09-13T09:09:25.175644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"121'558","reported_seq":5247,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.259654+0000","last_change":"2024-09-13T09:08:57.963388+0000","last_active":"2024-09-13T09:09:25.259654+0000","last_peered":"2024-09-13T09:09:25.259654+0000","last_clean":"2024-09-13T09:09:25.259654+0000","last_became_active":"2024-09-13T09:08:57.961197+0000","last_became_peered":"2024-09-13T09:08:57.961197+0000","last_unstale":"2024-09-13T09:09:25.259654+0000","last_undegraded":"2024-09-13T09:09:25.259654+0000","last_fullsized":"2024-09-13T09:09:25.259654+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:00.864515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":51047715,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":5923,"num_read_kb":21483,"num_write":720,"num_write_kb":6693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":6759140,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.179618+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:09:24.179618+0000","last_peered":"2024-09-13T09:09:24.179618+0000","last_clean":"2024-09-13T09:09:24.179618+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:09:24.179618+0000","last_undegraded":"2024-09-13T09:09:24.179618+0000","last_fullsized":"2024-09-13T09:09:24.179618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"125'366","reported_seq":2103,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.259474+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:09:25.259474+0000","last_peered":"2024-09-13T09:09:25.259474+0000","last_clean":"2024-09-13T09:09:25.259474+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:09:25.259474+0000","last_undegraded":"2024-09-13T09:09:25.259474+0000","last_fullsized":"2024-09-13T09:09:25.259474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":24744224,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":2502,"num_read_kb":13800,"num_write":348,"num_write_kb":5293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175520+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:09:25.175520+0000","last_peered":"2024-09-13T09:09:25.175520+0000","last_clean":"2024-09-13T09:09:25.175520+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:09:25.175520+0000","last_undegraded":"2024-09-13T09:09:25.175520+0000","last_fullsized":"2024-09-13T09:09:25.175520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"128'487","reported_seq":5694,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.183021+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:09:25.183021+0000","last_peered":"2024-09-13T09:09:25.183021+0000","last_clean":"2024-09-13T09:09:25.183021+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:09:25.183021+0000","last_undegraded":"2024-09-13T09:09:25.183021+0000","last_fullsized":"2024-09-13T09:09:25.183021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":50562806,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":6372,"num_read_kb":23846,"num_write":672,"num_write_kb":6606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":127,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:27.208301+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:09:27.208301+0000","last_peered":"2024-09-13T09:09:27.208301+0000","last_clean":"2024-09-13T09:09:27.208301+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:09:27.208301+0000","last_undegraded":"2024-09-13T09:09:27.208301+0000","last_fullsized":"2024-09-13T09:09:27.208301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"128'581","reported_seq":3299,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.186323+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:09:26.186323+0000","last_peered":"2024-09-13T09:09:26.186323+0000","last_clean":"2024-09-13T09:09:26.186323+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:09:26.186323+0000","last_undegraded":"2024-09-13T09:09:26.186323+0000","last_fullsized":"2024-09-13T09:09:26.186323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":42594179,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":16,"num_read":4375,"num_read_kb":19295,"num_write":789,"num_write_kb":10448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"128'377","reported_seq":2889,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.142932+0000","last_change":"2024-09-13T09:08:38.211675+0000","last_active":"2024-09-13T09:09:26.142932+0000","last_peered":"2024-09-13T09:09:26.142932+0000","last_clean":"2024-09-13T09:09:26.142932+0000","last_became_active":"2024-09-13T09:08:38.209914+0000","last_became_peered":"2024-09-13T09:08:38.209914+0000","last_unstale":"2024-09-13T09:09:26.142932+0000","last_undegraded":"2024-09-13T09:09:26.142932+0000","last_fullsized":"2024-09-13T09:09:26.142932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:19.947594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":80963318,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":3031,"num_read_kb":17308,"num_write":400,"num_write_kb":5551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":918446,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":127,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175322+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:09:25.175322+0000","last_peered":"2024-09-13T09:09:25.175322+0000","last_clean":"2024-09-13T09:09:25.175322+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:09:25.175322+0000","last_undegraded":"2024-09-13T09:09:25.175322+0000","last_fullsized":"2024-09-13T09:09:25.175322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"127'398","reported_seq":2981,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:27.597473+0000","last_change":"2024-09-13T09:08:57.959945+0000","last_active":"2024-09-13T09:09:27.597473+0000","last_peered":"2024-09-13T09:09:27.597473+0000","last_clean":"2024-09-13T09:09:27.597473+0000","last_became_active":"2024-09-13T09:08:57.957875+0000","last_became_peered":"2024-09-13T09:08:57.957875+0000","last_unstale":"2024-09-13T09:09:27.597473+0000","last_undegraded":"2024-09-13T09:09:27.597473+0000","last_fullsized":"2024-09-13T09:09:27.597473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.313991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":25275249,"num_objects":68,"num_object_clones":14,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":30,"num_read":3178,"num_read_kb":18973,"num_write":436,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3111448,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:27.208069+0000","last_change":"2024-09-13T09:08:54.932713+0000","last_active":"2024-09-13T09:09:27.208069+0000","last_peered":"2024-09-13T09:09:27.208069+0000","last_clean":"2024-09-13T09:09:27.208069+0000","last_became_active":"2024-09-13T09:08:54.931779+0000","last_became_peered":"2024-09-13T09:08:54.931779+0000","last_unstale":"2024-09-13T09:09:27.208069+0000","last_undegraded":"2024-09-13T09:09:27.208069+0000","last_fullsized":"2024-09-13T09:09:27.208069+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:42:37.917578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"127'334","reported_seq":1761,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.494155+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:09:24.494155+0000","last_peered":"2024-09-13T09:09:24.494155+0000","last_clean":"2024-09-13T09:09:24.494155+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:09:24.494155+0000","last_undegraded":"2024-09-13T09:09:24.494155+0000","last_fullsized":"2024-09-13T09:09:24.494155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":17541684,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":1714,"num_read_kb":13068,"num_write":356,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175106+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:09:25.175106+0000","last_peered":"2024-09-13T09:09:25.175106+0000","last_clean":"2024-09-13T09:09:25.175106+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:09:25.175106+0000","last_undegraded":"2024-09-13T09:09:25.175106+0000","last_fullsized":"2024-09-13T09:09:25.175106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"125'343","reported_seq":1997,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.179216+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:09:24.179216+0000","last_peered":"2024-09-13T09:09:24.179216+0000","last_clean":"2024-09-13T09:09:24.179216+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:09:24.179216+0000","last_undegraded":"2024-09-13T09:09:24.179216+0000","last_fullsized":"2024-09-13T09:09:24.179216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":16933977,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":2173,"num_read_kb":10013,"num_write":380,"num_write_kb":3720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":126,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175347+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:09:25.175347+0000","last_peered":"2024-09-13T09:09:25.175347+0000","last_clean":"2024-09-13T09:09:25.175347+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:09:25.175347+0000","last_undegraded":"2024-09-13T09:09:25.175347+0000","last_fullsized":"2024-09-13T09:09:25.175347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":254,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.195971+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:09:25.195971+0000","last_peered":"2024-09-13T09:09:25.195971+0000","last_clean":"2024-09-13T09:09:25.195971+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:09:25.195971+0000","last_undegraded":"2024-09-13T09:09:25.195971+0000","last_fullsized":"2024-09-13T09:09:25.195971+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"125'803","reported_seq":5467,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.259051+0000","last_change":"2024-09-13T09:08:57.966773+0000","last_active":"2024-09-13T09:09:25.259051+0000","last_peered":"2024-09-13T09:09:25.259051+0000","last_clean":"2024-09-13T09:09:25.259051+0000","last_became_active":"2024-09-13T09:08:57.964033+0000","last_became_peered":"2024-09-13T09:08:57.964033+0000","last_unstale":"2024-09-13T09:09:25.259051+0000","last_undegraded":"2024-09-13T09:09:25.259051+0000","last_fullsized":"2024-09-13T09:09:25.259051+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:54.967175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":57025170,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":6768,"num_read_kb":22892,"num_write":1224,"num_write_kb":11611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":4982043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'471","reported_seq":3043,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.181798+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:09:26.181798+0000","last_peered":"2024-09-13T09:09:26.181798+0000","last_clean":"2024-09-13T09:09:26.181798+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:09:26.181798+0000","last_undegraded":"2024-09-13T09:09:26.181798+0000","last_fullsized":"2024-09-13T09:09:26.181798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":48034015,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":4251,"num_read_kb":16076,"num_write":578,"num_write_kb":8946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"128'494","reported_seq":2620,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.259128+0000","last_change":"2024-09-13T09:08:21.089403+0000","last_active":"2024-09-13T09:09:25.259128+0000","last_peered":"2024-09-13T09:09:25.259128+0000","last_clean":"2024-09-13T09:09:25.259128+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:09:25.259128+0000","last_undegraded":"2024-09-13T09:09:25.259128+0000","last_fullsized":"2024-09-13T09:09:25.259128+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":83461235,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":3648,"num_read_kb":22312,"num_write":482,"num_write_kb":16962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"127'344","reported_seq":3387,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.317963+0000","last_change":"2024-09-13T09:08:57.963838+0000","last_active":"2024-09-13T09:09:24.317963+0000","last_peered":"2024-09-13T09:09:24.317963+0000","last_clean":"2024-09-13T09:09:24.317963+0000","last_became_active":"2024-09-13T09:08:57.961707+0000","last_became_peered":"2024-09-13T09:08:57.961707+0000","last_unstale":"2024-09-13T09:09:24.317963+0000","last_undegraded":"2024-09-13T09:09:24.317963+0000","last_fullsized":"2024-09-13T09:09:24.317963+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:58.156711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":38533428,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":17,"num_read":3407,"num_read_kb":33772,"num_write":375,"num_write_kb":17801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":8453655,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"125'281","reported_seq":16732,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175240+0000","last_change":"2024-09-13T09:08:57.961063+0000","last_active":"2024-09-13T09:09:25.175240+0000","last_peered":"2024-09-13T09:09:25.175240+0000","last_clean":"2024-09-13T09:09:25.175240+0000","last_became_active":"2024-09-13T09:08:57.959990+0000","last_became_peered":"2024-09-13T09:08:57.959990+0000","last_unstale":"2024-09-13T09:09:25.175240+0000","last_undegraded":"2024-09-13T09:09:25.175240+0000","last_fullsized":"2024-09-13T09:09:25.175240+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:21.370554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":49991740,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":16411,"num_read_kb":27526,"num_write":315,"num_write_kb":14368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":26451574,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"129'324","reported_seq":2674,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.183356+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:09:26.183356+0000","last_peered":"2024-09-13T09:09:26.183356+0000","last_clean":"2024-09-13T09:09:26.183356+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:09:26.183356+0000","last_undegraded":"2024-09-13T09:09:26.183356+0000","last_fullsized":"2024-09-13T09:09:26.183356+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":32725381,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":2833,"num_read_kb":8928,"num_write":318,"num_write_kb":4714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"125'716","reported_seq":6305,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.602103+0000","last_change":"2024-09-13T09:08:38.205741+0000","last_active":"2024-09-13T09:09:25.602103+0000","last_peered":"2024-09-13T09:09:25.602103+0000","last_clean":"2024-09-13T09:09:25.602103+0000","last_became_active":"2024-09-13T09:08:38.205194+0000","last_became_peered":"2024-09-13T09:08:38.205194+0000","last_unstale":"2024-09-13T09:09:25.602103+0000","last_undegraded":"2024-09-13T09:09:25.602103+0000","last_fullsized":"2024-09-13T09:09:25.602103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:14.407799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":50364519,"num_objects":60,"num_object_clones":15,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":20,"num_read":8337,"num_read_kb":22368,"num_write":1102,"num_write_kb":5107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"128'374","reported_seq":4544,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.132173+0000","last_change":"2024-09-13T09:09:16.093748+0000","last_active":"2024-09-13T09:09:26.132173+0000","last_peered":"2024-09-13T09:09:26.132173+0000","last_clean":"2024-09-13T09:09:26.132173+0000","last_became_active":"2024-09-13T09:09:16.091543+0000","last_became_peered":"2024-09-13T09:09:16.091543+0000","last_unstale":"2024-09-13T09:09:26.132173+0000","last_undegraded":"2024-09-13T09:09:26.132173+0000","last_fullsized":"2024-09-13T09:09:26.132173+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:12.463442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":44392247,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":21,"num_read":4460,"num_read_kb":16367,"num_write":359,"num_write_kb":6348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":9699455,"num_keys_recovered":77,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"128'435","reported_seq":2422,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.102350+0000","last_change":"2024-09-13T09:08:09.863895+0000","last_active":"2024-09-13T09:09:26.102350+0000","last_peered":"2024-09-13T09:09:26.102350+0000","last_clean":"2024-09-13T09:09:26.102350+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:09:26.102350+0000","last_undegraded":"2024-09-13T09:09:26.102350+0000","last_fullsized":"2024-09-13T09:09:26.102350+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":60314846,"num_objects":62,"num_object_clones":7,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":22,"num_read":2588,"num_read_kb":16895,"num_write":432,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"128'989","reported_seq":3627,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.074512+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:09:26.074512+0000","last_peered":"2024-09-13T09:09:26.074512+0000","last_clean":"2024-09-13T09:09:26.074512+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:09:26.074512+0000","last_undegraded":"2024-09-13T09:09:26.074512+0000","last_fullsized":"2024-09-13T09:09:26.074512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":40752047,"num_objects":57,"num_object_clones":9,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":16,"num_read":6891,"num_read_kb":11564,"num_write":1494,"num_write_kb":5302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"127'613","reported_seq":4122,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.399330+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:09:24.399330+0000","last_peered":"2024-09-13T09:09:24.399330+0000","last_clean":"2024-09-13T09:09:24.399330+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:09:24.399330+0000","last_undegraded":"2024-09-13T09:09:24.399330+0000","last_fullsized":"2024-09-13T09:09:24.399330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":41713937,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":15,"num_read":5518,"num_read_kb":13758,"num_write":898,"num_write_kb":10316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"128'485","reported_seq":3371,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.109645+0000","last_change":"2024-09-13T09:09:04.006383+0000","last_active":"2024-09-13T09:09:26.109645+0000","last_peered":"2024-09-13T09:09:26.109645+0000","last_clean":"2024-09-13T09:09:26.109645+0000","last_became_active":"2024-09-13T09:09:04.005219+0000","last_became_peered":"2024-09-13T09:09:04.005219+0000","last_unstale":"2024-09-13T09:09:26.109645+0000","last_undegraded":"2024-09-13T09:09:26.109645+0000","last_fullsized":"2024-09-13T09:09:26.109645+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:56.239198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":25027959,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":27,"num_read":3107,"num_read_kb":12625,"num_write":544,"num_write_kb":7274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1558811262,"num_objects":1802,"num_object_clones":390,"num_object_copies":3604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1802,"num_whiteouts":662,"num_read":195988,"num_read_kb":664546,"num_write":22309,"num_write_kb":327051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1769,"num_bytes_recovered":174938849,"num_keys_recovered":405,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401698816,"data_stored":441903608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17608,"ondisk_log_size":17608,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509473,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":15,"seq":64424509473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147156,"kb_used_data":71752,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94224684,"statfs":{"total":96636764160,"available":96486076416,"internally_reserved":0,"allocated":73474048,"data_stored":82682254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.115,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.27200000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.121,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.071,"5min":1.444,"15min":1.444},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59399999999999997,"15min":0.57199999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85199999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2989999999999999,"5min":1.343,"15min":1.343},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.873,"15min":0.86499999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.151},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.878}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.92200000000000004,"15min":0.90600000000000003},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.075},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.92500000000000004,"15min":0.92100000000000004},"min":{"1min":0.52600000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.379}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.125},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.506}]}]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131824,"kb_used_data":56420,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94240016,"statfs":{"total":96636764160,"available":96501776384,"internally_reserved":0,"allocated":57774080,"data_stored":59183988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73999999999999999,"15min":0.751},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80600000000000005,"15min":0.79900000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.88100000000000001,"15min":0.91400000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.0109999999999999,"15min":1.0389999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.95499999999999996},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.492,"5min":1.651,"15min":1.651},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.98299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.95999999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3049999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.107,"15min":1.131},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5840000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":0.93600000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86116,"kb_used_data":59100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94285724,"statfs":{"total":96636764160,"available":96548581376,"internally_reserved":0,"allocated":60518400,"data_stored":64892073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.625,"15min":0.64400000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.995,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.036,"5min":1.214,"15min":1.214},"last":0.98399999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96899999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.749},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.407,"15min":1.407},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.121,"5min":1.415,"15min":1.415},"last":0.872},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.92200000000000004,"15min":0.94199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2270000000000001,"5min":1.448,"15min":1.448},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96599999999999997,"15min":0.98599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.96299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2010000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.052}]},{"osd":6,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.47399999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.353,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.878,"5min":1.0309999999999999,"15min":1.0569999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3460000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.083}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.1180000000000001}]}]},{"osd":0,"up_from":0,"seq":64424509474,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69020,"kb_used_data":37908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94302820,"statfs":{"total":96636764160,"available":96566087680,"internally_reserved":0,"allocated":38817792,"data_stored":44941130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77700000000000002,"15min":0.76400000000000001},"min":{"1min":0.47499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.82199999999999995,"15min":0.79700000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.92900000000000005,"15min":0.92800000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":0.97199999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":0.94899999999999995,"15min":0.90100000000000002},"min":{"1min":0.53700000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":8.3499999999999996,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.206,"5min":0.98199999999999998,"15min":0.94399999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":8.2989999999999995,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.99399999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.95099999999999996,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.472,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.92300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.038}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.218,"5min":1.7,"15min":1.7},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.085,"15min":1.093},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0700000000000001,"15min":1.0720000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.091}]}]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104772,"kb_used_data":69556,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94267068,"statfs":{"total":96636764160,"available":96529477632,"internally_reserved":0,"allocated":71225344,"data_stored":78149784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.115,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":1.3,"15min":1.3},"last":0.86599999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.84799999999999998,"15min":0.81999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.90900000000000003,"15min":0.91600000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.399}]},{"osd":3,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.873,"15min":0.85299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.88900000000000001,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.248}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.5269999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.008,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.1020000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.206},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.3400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122236,"kb_used_data":64884,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94249604,"statfs":{"total":96636764160,"available":96511594496,"internally_reserved":0,"allocated":66441216,"data_stored":68714546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":1.1200000000000001,"15min":1.179},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0489999999999999,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.085,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.218,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.1080000000000001,"15min":1.159},"min":{"1min":0.54900000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.151,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.026}]},{"osd":3,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4910000000000001,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.96999999999999997,"15min":0.97799999999999998},"min":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.585},"last":0.83899999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0169999999999999,"15min":1.0309999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.123},{"interface":"front","average":{"1min":1.006,"5min":1.357,"15min":1.415},"min":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3799999999999999,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.012,"5min":1.0800000000000001,"15min":1.091},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4710000000000001,"5min":1.899,"15min":1.899},"last":1.1799999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.2450000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6295552,"data_stored":7615357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58417152,"data_stored":63600089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":56336384,"data_stored":58545765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":72056832,"data_stored":82069291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69713920,"data_stored":77441026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":66441216,"data_stored":69531554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":37310464,"data_stored":44232514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":35127296,"data_stored":38868012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:09:30.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:09:30.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:30.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:30.719 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:30.737 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:30.737 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:30.749 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:30.744+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:30.755 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:30.755 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:30.771 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:30.771 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:30.791 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:30.791 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:30.812 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:30.829 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:30.829 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:30.830 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:30.831 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:30.833 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:30.828+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:30.844 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:30.851 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:30.851 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:30.937 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:30.932+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:31.014 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:09:31.015 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:09:31.021 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:31.021 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:31.032 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-13T09:09:30.210833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1559401649,"num_objects":1806,"num_object_clones":390,"num_object_copies":3612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1806,"num_whiteouts":662,"num_read":196104,"num_read_kb":664646,"num_write":22470,"num_write_kb":329469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":175529217,"num_keys_recovered":405,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17728,"ondisk_log_size":17728,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":661124,"kb_used_data":359620,"kb_used_omap":9,"kb_used_meta":301430,"kb_avail":565569916,"statfs":{"total":579820584960,"available":579143593984,"internally_reserved":0,"allocated":368250880,"data_stored":398563775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":308664999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10589861,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":7,"num_read":2942,"num_read_kb":13202,"num_write":379,"num_write_kb":2852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.038058"},"pg_stats":[{"pgid":"3.1f","version":"130'494","reported_seq":5374,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:28.146211+0000","last_change":"2024-09-13T09:07:53.864866+0000","last_active":"2024-09-13T09:09:28.146211+0000","last_peered":"2024-09-13T09:09:28.146211+0000","last_clean":"2024-09-13T09:09:28.146211+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:09:28.146211+0000","last_undegraded":"2024-09-13T09:09:28.146211+0000","last_fullsized":"2024-09-13T09:09:28.146211+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071480600000000001,"stat_sum":{"num_bytes":40723036,"num_objects":65,"num_object_clones":16,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":28,"num_read":5902,"num_read_kb":20167,"num_write":631,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"130'1060","reported_seq":4403,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:28.135341+0000","last_change":"2024-09-13T09:07:15.854096+0000","last_active":"2024-09-13T09:09:28.135341+0000","last_peered":"2024-09-13T09:09:28.135341+0000","last_clean":"2024-09-13T09:09:28.135341+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:09:28.135341+0000","last_undegraded":"2024-09-13T09:09:28.135341+0000","last_fullsized":"2024-09-13T09:09:28.135341+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1060,"log_dups_size":0,"ondisk_log_size":1060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000389711,"stat_sum":{"num_bytes":48051095,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":26,"num_read":8453,"num_read_kb":18493,"num_write":1447,"num_write_kb":5996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"127'1332","reported_seq":5059,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175441+0000","last_change":"2024-09-13T09:08:38.210944+0000","last_active":"2024-09-13T09:09:25.175441+0000","last_peered":"2024-09-13T09:09:25.175441+0000","last_clean":"2024-09-13T09:09:25.175441+0000","last_became_active":"2024-09-13T09:08:38.208191+0000","last_became_peered":"2024-09-13T09:08:38.208191+0000","last_unstale":"2024-09-13T09:09:25.175441+0000","last_undegraded":"2024-09-13T09:09:25.175441+0000","last_fullsized":"2024-09-13T09:09:25.175441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1332,"log_dups_size":0,"ondisk_log_size":1332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:04.686300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001610511,"stat_sum":{"num_bytes":54547238,"num_objects":68,"num_object_clones":17,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":31,"num_read":8690,"num_read_kb":33724,"num_write":2192,"num_write_kb":17061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"129'572","reported_seq":3382,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.184373+0000","last_change":"2024-09-13T09:08:38.212078+0000","last_active":"2024-09-13T09:09:26.184373+0000","last_peered":"2024-09-13T09:09:26.184373+0000","last_clean":"2024-09-13T09:09:26.184373+0000","last_became_active":"2024-09-13T09:08:38.209451+0000","last_became_peered":"2024-09-13T09:08:38.209451+0000","last_unstale":"2024-09-13T09:09:26.184373+0000","last_undegraded":"2024-09-13T09:09:26.184373+0000","last_fullsized":"2024-09-13T09:09:26.184373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:59.672566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099604100000000003,"stat_sum":{"num_bytes":79776230,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":19,"num_read":5156,"num_read_kb":34819,"num_write":801,"num_write_kb":15621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"127'577","reported_seq":21506,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.399055+0000","last_change":"2024-09-13T09:08:38.212581+0000","last_active":"2024-09-13T09:09:24.399055+0000","last_peered":"2024-09-13T09:09:24.399055+0000","last_clean":"2024-09-13T09:09:24.399055+0000","last_became_active":"2024-09-13T09:08:38.210333+0000","last_became_peered":"2024-09-13T09:08:38.210333+0000","last_unstale":"2024-09-13T09:09:24.399055+0000","last_undegraded":"2024-09-13T09:09:24.399055+0000","last_fullsized":"2024-09-13T09:09:24.399055+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:53.493070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088828500000000005,"stat_sum":{"num_bytes":94460825,"num_objects":61,"num_object_clones":12,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":19,"num_read":22971,"num_read_kb":47577,"num_write":676,"num_write_kb":18535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"127'574","reported_seq":5403,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.170959+0000","last_change":"2024-09-13T09:08:09.864519+0000","last_active":"2024-09-13T09:09:26.170959+0000","last_peered":"2024-09-13T09:09:26.170959+0000","last_clean":"2024-09-13T09:09:26.170959+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:09:26.170959+0000","last_undegraded":"2024-09-13T09:09:26.170959+0000","last_fullsized":"2024-09-13T09:09:26.170959+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087810799999999995,"stat_sum":{"num_bytes":62903922,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":26,"num_read":7305,"num_read_kb":23969,"num_write":758,"num_write_kb":19650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"125'323","reported_seq":1676,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.474591+0000","last_change":"2024-09-13T09:07:50.944073+0000","last_active":"2024-09-13T09:09:24.474591+0000","last_peered":"2024-09-13T09:09:24.474591+0000","last_clean":"2024-09-13T09:09:24.474591+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:09:24.474591+0000","last_undegraded":"2024-09-13T09:09:24.474591+0000","last_fullsized":"2024-09-13T09:09:24.474591+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":323,"log_dups_size":0,"ondisk_log_size":323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011744209999999999,"stat_sum":{"num_bytes":15863077,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":1622,"num_read_kb":7909,"num_write":307,"num_write_kb":7298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"130'579","reported_seq":2916,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:28.134903+0000","last_change":"2024-09-13T09:07:15.854397+0000","last_active":"2024-09-13T09:09:28.134903+0000","last_peered":"2024-09-13T09:09:28.134903+0000","last_clean":"2024-09-13T09:09:28.134903+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:09:28.134903+0000","last_undegraded":"2024-09-13T09:09:28.134903+0000","last_fullsized":"2024-09-13T09:09:28.134903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064237400000000003,"stat_sum":{"num_bytes":47319500,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":26,"num_read":4163,"num_read_kb":14101,"num_write":675,"num_write_kb":10427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"131'532","reported_seq":5194,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T09:09:28.195785+0000","last_change":"2024-09-13T09:07:15.854763+0000","last_active":"2024-09-13T09:09:28.195785+0000","last_peered":"2024-09-13T09:09:28.195785+0000","last_clean":"2024-09-13T09:09:28.195785+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:09:28.195785+0000","last_undegraded":"2024-09-13T09:09:28.195785+0000","last_fullsized":"2024-09-13T09:09:28.195785+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090909900000000002,"stat_sum":{"num_bytes":55833489,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":17,"num_read":6428,"num_read_kb":30595,"num_write":549,"num_write_kb":18247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"128'641","reported_seq":1705,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.135087+0000","last_change":"2024-09-13T09:07:15.859376+0000","last_active":"2024-09-13T09:09:26.135087+0000","last_peered":"2024-09-13T09:09:26.135087+0000","last_clean":"2024-09-13T09:09:26.135087+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:09:26.135087+0000","last_undegraded":"2024-09-13T09:09:26.135087+0000","last_fullsized":"2024-09-13T09:09:26.135087+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":641,"log_dups_size":0,"ondisk_log_size":641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070966599999999996,"stat_sum":{"num_bytes":36375948,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":2806,"num_read_kb":13605,"num_write":920,"num_write_kb":9801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"128'569","reported_seq":3713,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.211835+0000","last_change":"2024-09-13T09:08:38.222834+0000","last_active":"2024-09-13T09:09:25.211835+0000","last_peered":"2024-09-13T09:09:25.211835+0000","last_clean":"2024-09-13T09:09:25.211835+0000","last_became_active":"2024-09-13T09:08:38.221484+0000","last_became_peered":"2024-09-13T09:08:38.221484+0000","last_unstale":"2024-09-13T09:09:25.211835+0000","last_undegraded":"2024-09-13T09:09:25.211835+0000","last_fullsized":"2024-09-13T09:09:25.211835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:04.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040681700000000002,"stat_sum":{"num_bytes":93809147,"num_objects":66,"num_object_clones":20,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":21,"num_read":4632,"num_read_kb":23977,"num_write":589,"num_write_kb":14829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"128'582","reported_seq":18691,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.185120+0000","last_change":"2024-09-13T09:09:24.174071+0000","last_active":"2024-09-13T09:09:26.185120+0000","last_peered":"2024-09-13T09:09:26.185120+0000","last_clean":"2024-09-13T09:09:26.185120+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:09:26.185120+0000","last_undegraded":"2024-09-13T09:09:26.185120+0000","last_fullsized":"2024-09-13T09:09:26.185120+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015917010000000001,"stat_sum":{"num_bytes":47148079,"num_objects":60,"num_object_clones":15,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":18,"num_read":20373,"num_read_kb":32741,"num_write":542,"num_write_kb":9900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":133,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175644+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:09:25.175644+0000","last_peered":"2024-09-13T09:09:25.175644+0000","last_clean":"2024-09-13T09:09:25.175644+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:09:25.175644+0000","last_undegraded":"2024-09-13T09:09:25.175644+0000","last_fullsized":"2024-09-13T09:09:25.175644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"121'558","reported_seq":5247,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.259654+0000","last_change":"2024-09-13T09:08:57.963388+0000","last_active":"2024-09-13T09:09:25.259654+0000","last_peered":"2024-09-13T09:09:25.259654+0000","last_clean":"2024-09-13T09:09:25.259654+0000","last_became_active":"2024-09-13T09:08:57.961197+0000","last_became_peered":"2024-09-13T09:08:57.961197+0000","last_unstale":"2024-09-13T09:09:25.259654+0000","last_undegraded":"2024-09-13T09:09:25.259654+0000","last_fullsized":"2024-09-13T09:09:25.259654+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:00.864515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056408999999999995,"stat_sum":{"num_bytes":51047715,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":5923,"num_read_kb":21483,"num_write":720,"num_write_kb":6693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":6759140,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.179618+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:09:24.179618+0000","last_peered":"2024-09-13T09:09:24.179618+0000","last_clean":"2024-09-13T09:09:24.179618+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:09:24.179618+0000","last_undegraded":"2024-09-13T09:09:24.179618+0000","last_fullsized":"2024-09-13T09:09:24.179618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"125'366","reported_seq":2103,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.259474+0000","last_change":"2024-09-13T09:07:16.108262+0000","last_active":"2024-09-13T09:09:25.259474+0000","last_peered":"2024-09-13T09:09:25.259474+0000","last_clean":"2024-09-13T09:09:25.259474+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:09:25.259474+0000","last_undegraded":"2024-09-13T09:09:25.259474+0000","last_fullsized":"2024-09-13T09:09:25.259474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055128999999999996,"stat_sum":{"num_bytes":24744224,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":21,"num_read":2502,"num_read_kb":13800,"num_write":348,"num_write_kb":5293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":125,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175520+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:09:25.175520+0000","last_peered":"2024-09-13T09:09:25.175520+0000","last_clean":"2024-09-13T09:09:25.175520+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:09:25.175520+0000","last_undegraded":"2024-09-13T09:09:25.175520+0000","last_fullsized":"2024-09-13T09:09:25.175520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"128'487","reported_seq":5694,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.183021+0000","last_change":"2024-09-13T09:07:16.110219+0000","last_active":"2024-09-13T09:09:25.183021+0000","last_peered":"2024-09-13T09:09:25.183021+0000","last_clean":"2024-09-13T09:09:25.183021+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:09:25.183021+0000","last_undegraded":"2024-09-13T09:09:25.183021+0000","last_fullsized":"2024-09-13T09:09:25.183021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076637500000000002,"stat_sum":{"num_bytes":50562806,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":14,"num_read":6372,"num_read_kb":23846,"num_write":672,"num_write_kb":6606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":127,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:27.208301+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:09:27.208301+0000","last_peered":"2024-09-13T09:09:27.208301+0000","last_clean":"2024-09-13T09:09:27.208301+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:09:27.208301+0000","last_undegraded":"2024-09-13T09:09:27.208301+0000","last_fullsized":"2024-09-13T09:09:27.208301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"128'581","reported_seq":3299,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.186323+0000","last_change":"2024-09-13T09:07:15.859774+0000","last_active":"2024-09-13T09:09:26.186323+0000","last_peered":"2024-09-13T09:09:26.186323+0000","last_clean":"2024-09-13T09:09:26.186323+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:09:26.186323+0000","last_undegraded":"2024-09-13T09:09:26.186323+0000","last_fullsized":"2024-09-13T09:09:26.186323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096591099999999996,"stat_sum":{"num_bytes":42594179,"num_objects":53,"num_object_clones":9,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":16,"num_read":4375,"num_read_kb":19295,"num_write":789,"num_write_kb":10448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"128'377","reported_seq":2889,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.142932+0000","last_change":"2024-09-13T09:08:38.211675+0000","last_active":"2024-09-13T09:09:26.142932+0000","last_peered":"2024-09-13T09:09:26.142932+0000","last_clean":"2024-09-13T09:09:26.142932+0000","last_became_active":"2024-09-13T09:08:38.209914+0000","last_became_peered":"2024-09-13T09:08:38.209914+0000","last_unstale":"2024-09-13T09:09:26.142932+0000","last_undegraded":"2024-09-13T09:09:26.142932+0000","last_fullsized":"2024-09-13T09:09:26.142932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:19.947594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000403824,"stat_sum":{"num_bytes":80963318,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":30,"num_read":3031,"num_read_kb":17308,"num_write":400,"num_write_kb":5551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":918446,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":127,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175322+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:09:25.175322+0000","last_peered":"2024-09-13T09:09:25.175322+0000","last_clean":"2024-09-13T09:09:25.175322+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:09:25.175322+0000","last_undegraded":"2024-09-13T09:09:25.175322+0000","last_fullsized":"2024-09-13T09:09:25.175322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"127'398","reported_seq":2981,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:27.597473+0000","last_change":"2024-09-13T09:08:57.959945+0000","last_active":"2024-09-13T09:09:27.597473+0000","last_peered":"2024-09-13T09:09:27.597473+0000","last_clean":"2024-09-13T09:09:27.597473+0000","last_became_active":"2024-09-13T09:08:57.957875+0000","last_became_peered":"2024-09-13T09:08:57.957875+0000","last_unstale":"2024-09-13T09:09:27.597473+0000","last_undegraded":"2024-09-13T09:09:27.597473+0000","last_fullsized":"2024-09-13T09:09:27.597473+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.313991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055704799999999998,"stat_sum":{"num_bytes":25275249,"num_objects":68,"num_object_clones":14,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":30,"num_read":3178,"num_read_kb":18973,"num_write":436,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3111448,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T09:09:27.208069+0000","last_change":"2024-09-13T09:08:54.932713+0000","last_active":"2024-09-13T09:09:27.208069+0000","last_peered":"2024-09-13T09:09:27.208069+0000","last_clean":"2024-09-13T09:09:27.208069+0000","last_became_active":"2024-09-13T09:08:54.931779+0000","last_became_peered":"2024-09-13T09:08:54.931779+0000","last_unstale":"2024-09-13T09:09:27.208069+0000","last_undegraded":"2024-09-13T09:09:27.208069+0000","last_fullsized":"2024-09-13T09:09:27.208069+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:42:37.917578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"127'334","reported_seq":1761,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.494155+0000","last_change":"2024-09-13T09:07:16.110211+0000","last_active":"2024-09-13T09:09:24.494155+0000","last_peered":"2024-09-13T09:09:24.494155+0000","last_clean":"2024-09-13T09:09:24.494155+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:09:24.494155+0000","last_undegraded":"2024-09-13T09:09:24.494155+0000","last_fullsized":"2024-09-13T09:09:24.494155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100832,"stat_sum":{"num_bytes":17541684,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":1714,"num_read_kb":13068,"num_write":356,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":125,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175106+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:09:25.175106+0000","last_peered":"2024-09-13T09:09:25.175106+0000","last_clean":"2024-09-13T09:09:25.175106+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:09:25.175106+0000","last_undegraded":"2024-09-13T09:09:25.175106+0000","last_fullsized":"2024-09-13T09:09:25.175106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"125'343","reported_seq":1997,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.179216+0000","last_change":"2024-09-13T09:07:16.108114+0000","last_active":"2024-09-13T09:09:24.179216+0000","last_peered":"2024-09-13T09:09:24.179216+0000","last_clean":"2024-09-13T09:09:24.179216+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:09:24.179216+0000","last_undegraded":"2024-09-13T09:09:24.179216+0000","last_fullsized":"2024-09-13T09:09:24.179216+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045947700000000002,"stat_sum":{"num_bytes":16933977,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":2173,"num_read_kb":10013,"num_write":380,"num_write_kb":3720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":126,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175347+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:09:25.175347+0000","last_peered":"2024-09-13T09:09:25.175347+0000","last_clean":"2024-09-13T09:09:25.175347+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:09:25.175347+0000","last_undegraded":"2024-09-13T09:09:25.175347+0000","last_fullsized":"2024-09-13T09:09:25.175347+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":254,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.195971+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:09:25.195971+0000","last_peered":"2024-09-13T09:09:25.195971+0000","last_clean":"2024-09-13T09:09:25.195971+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:09:25.195971+0000","last_undegraded":"2024-09-13T09:09:25.195971+0000","last_fullsized":"2024-09-13T09:09:25.195971+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"125'803","reported_seq":5467,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.259051+0000","last_change":"2024-09-13T09:08:57.966773+0000","last_active":"2024-09-13T09:09:25.259051+0000","last_peered":"2024-09-13T09:09:25.259051+0000","last_clean":"2024-09-13T09:09:25.259051+0000","last_became_active":"2024-09-13T09:08:57.964033+0000","last_became_peered":"2024-09-13T09:08:57.964033+0000","last_unstale":"2024-09-13T09:09:25.259051+0000","last_undegraded":"2024-09-13T09:09:25.259051+0000","last_fullsized":"2024-09-13T09:09:25.259051+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:54.967175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034875999999999999,"stat_sum":{"num_bytes":57025170,"num_objects":53,"num_object_clones":11,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":6768,"num_read_kb":22892,"num_write":1224,"num_write_kb":11611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":4982043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"129'471","reported_seq":3043,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.181798+0000","last_change":"2024-09-13T09:07:15.879561+0000","last_active":"2024-09-13T09:09:26.181798+0000","last_peered":"2024-09-13T09:09:26.181798+0000","last_clean":"2024-09-13T09:09:26.181798+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:09:26.181798+0000","last_undegraded":"2024-09-13T09:09:26.181798+0000","last_fullsized":"2024-09-13T09:09:26.181798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001278354,"stat_sum":{"num_bytes":48034015,"num_objects":44,"num_object_clones":6,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":4251,"num_read_kb":16076,"num_write":578,"num_write_kb":8946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"128'494","reported_seq":2620,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.259128+0000","last_change":"2024-09-13T09:08:21.089403+0000","last_active":"2024-09-13T09:09:25.259128+0000","last_peered":"2024-09-13T09:09:25.259128+0000","last_clean":"2024-09-13T09:09:25.259128+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:09:25.259128+0000","last_undegraded":"2024-09-13T09:09:25.259128+0000","last_fullsized":"2024-09-13T09:09:25.259128+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001710888,"stat_sum":{"num_bytes":83461235,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":3648,"num_read_kb":22312,"num_write":482,"num_write_kb":16962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"127'344","reported_seq":3387,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.317963+0000","last_change":"2024-09-13T09:08:57.963838+0000","last_active":"2024-09-13T09:09:24.317963+0000","last_peered":"2024-09-13T09:09:24.317963+0000","last_clean":"2024-09-13T09:09:24.317963+0000","last_became_active":"2024-09-13T09:08:57.961707+0000","last_became_peered":"2024-09-13T09:08:57.961707+0000","last_unstale":"2024-09-13T09:09:24.317963+0000","last_undegraded":"2024-09-13T09:09:24.317963+0000","last_fullsized":"2024-09-13T09:09:24.317963+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:58.156711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045884499999999999,"stat_sum":{"num_bytes":38533428,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":17,"num_read":3407,"num_read_kb":33772,"num_write":375,"num_write_kb":17801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":8453655,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"125'281","reported_seq":16732,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.175240+0000","last_change":"2024-09-13T09:08:57.961063+0000","last_active":"2024-09-13T09:09:25.175240+0000","last_peered":"2024-09-13T09:09:25.175240+0000","last_clean":"2024-09-13T09:09:25.175240+0000","last_became_active":"2024-09-13T09:08:57.959990+0000","last_became_peered":"2024-09-13T09:08:57.959990+0000","last_unstale":"2024-09-13T09:09:25.175240+0000","last_undegraded":"2024-09-13T09:09:25.175240+0000","last_fullsized":"2024-09-13T09:09:25.175240+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:21.370554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062670800000000002,"stat_sum":{"num_bytes":49991740,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":22,"num_read":16411,"num_read_kb":27526,"num_write":315,"num_write_kb":14368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":26451574,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"129'324","reported_seq":2674,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.183356+0000","last_change":"2024-09-13T09:07:50.989962+0000","last_active":"2024-09-13T09:09:26.183356+0000","last_peered":"2024-09-13T09:09:26.183356+0000","last_clean":"2024-09-13T09:09:26.183356+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:09:26.183356+0000","last_undegraded":"2024-09-13T09:09:26.183356+0000","last_fullsized":"2024-09-13T09:09:26.183356+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001872255,"stat_sum":{"num_bytes":32725381,"num_objects":51,"num_object_clones":10,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":20,"num_read":2833,"num_read_kb":8928,"num_write":318,"num_write_kb":4714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"125'716","reported_seq":6305,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:25.602103+0000","last_change":"2024-09-13T09:08:38.205741+0000","last_active":"2024-09-13T09:09:25.602103+0000","last_peered":"2024-09-13T09:09:25.602103+0000","last_clean":"2024-09-13T09:09:25.602103+0000","last_became_active":"2024-09-13T09:08:38.205194+0000","last_became_peered":"2024-09-13T09:08:38.205194+0000","last_unstale":"2024-09-13T09:09:25.602103+0000","last_undegraded":"2024-09-13T09:09:25.602103+0000","last_fullsized":"2024-09-13T09:09:25.602103+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":716,"log_dups_size":0,"ondisk_log_size":716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:14.407799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061760899999999997,"stat_sum":{"num_bytes":50364519,"num_objects":60,"num_object_clones":15,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":20,"num_read":8337,"num_read_kb":22368,"num_write":1102,"num_write_kb":5107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"128'374","reported_seq":4544,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.132173+0000","last_change":"2024-09-13T09:09:16.093748+0000","last_active":"2024-09-13T09:09:26.132173+0000","last_peered":"2024-09-13T09:09:26.132173+0000","last_clean":"2024-09-13T09:09:26.132173+0000","last_became_active":"2024-09-13T09:09:16.091543+0000","last_became_peered":"2024-09-13T09:09:16.091543+0000","last_unstale":"2024-09-13T09:09:26.132173+0000","last_undegraded":"2024-09-13T09:09:26.132173+0000","last_fullsized":"2024-09-13T09:09:26.132173+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:12.463442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053550099999999997,"stat_sum":{"num_bytes":44392247,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":21,"num_read":4460,"num_read_kb":16367,"num_write":359,"num_write_kb":6348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":9699455,"num_keys_recovered":77,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"128'435","reported_seq":2422,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.102350+0000","last_change":"2024-09-13T09:08:09.863895+0000","last_active":"2024-09-13T09:09:26.102350+0000","last_peered":"2024-09-13T09:09:26.102350+0000","last_clean":"2024-09-13T09:09:26.102350+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:09:26.102350+0000","last_undegraded":"2024-09-13T09:09:26.102350+0000","last_fullsized":"2024-09-13T09:09:26.102350+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054908699999999997,"stat_sum":{"num_bytes":60314846,"num_objects":62,"num_object_clones":7,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":22,"num_read":2588,"num_read_kb":16895,"num_write":432,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"128'989","reported_seq":3627,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.074512+0000","last_change":"2024-09-13T09:07:15.879538+0000","last_active":"2024-09-13T09:09:26.074512+0000","last_peered":"2024-09-13T09:09:26.074512+0000","last_clean":"2024-09-13T09:09:26.074512+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:09:26.074512+0000","last_undegraded":"2024-09-13T09:09:26.074512+0000","last_fullsized":"2024-09-13T09:09:26.074512+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":989,"log_dups_size":0,"ondisk_log_size":989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001300058,"stat_sum":{"num_bytes":40752047,"num_objects":57,"num_object_clones":9,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":16,"num_read":6891,"num_read_kb":11564,"num_write":1494,"num_write_kb":5302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"127'613","reported_seq":4122,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-13T09:09:24.399330+0000","last_change":"2024-09-13T09:07:16.109829+0000","last_active":"2024-09-13T09:09:24.399330+0000","last_peered":"2024-09-13T09:09:24.399330+0000","last_clean":"2024-09-13T09:09:24.399330+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:09:24.399330+0000","last_undegraded":"2024-09-13T09:09:24.399330+0000","last_fullsized":"2024-09-13T09:09:24.399330+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062040100000000003,"stat_sum":{"num_bytes":41713937,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":15,"num_read":5518,"num_read_kb":13758,"num_write":898,"num_write_kb":10316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"128'485","reported_seq":3371,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-13T09:09:26.109645+0000","last_change":"2024-09-13T09:09:04.006383+0000","last_active":"2024-09-13T09:09:26.109645+0000","last_peered":"2024-09-13T09:09:26.109645+0000","last_clean":"2024-09-13T09:09:26.109645+0000","last_became_active":"2024-09-13T09:09:04.005219+0000","last_became_peered":"2024-09-13T09:09:04.005219+0000","last_unstale":"2024-09-13T09:09:26.109645+0000","last_undegraded":"2024-09-13T09:09:26.109645+0000","last_fullsized":"2024-09-13T09:09:26.109645+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:56.239198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001711366,"stat_sum":{"num_bytes":25027959,"num_objects":67,"num_object_clones":14,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":27,"num_read":3107,"num_read_kb":12625,"num_write":544,"num_write_kb":7274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1558811262,"num_objects":1802,"num_object_clones":390,"num_object_copies":3604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1802,"num_whiteouts":662,"num_read":195988,"num_read_kb":664546,"num_write":22309,"num_write_kb":327051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1769,"num_bytes_recovered":174938849,"num_keys_recovered":405,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401698816,"data_stored":441903608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17608,"ondisk_log_size":17608,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509473,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":15,"seq":64424509473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147156,"kb_used_data":71752,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94224684,"statfs":{"total":96636764160,"available":96486076416,"internally_reserved":0,"allocated":73474048,"data_stored":82682254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.115,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.27200000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.121,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.071,"5min":1.444,"15min":1.444},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59399999999999997,"15min":0.57199999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85199999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2989999999999999,"5min":1.343,"15min":1.343},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.873,"15min":0.86499999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.151},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.878}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.92200000000000004,"15min":0.90600000000000003},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.075},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.92500000000000004,"15min":0.92100000000000004},"min":{"1min":0.52600000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":1.379}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.125},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.506}]}]},{"osd":2,"up_from":15,"seq":64424509472,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131824,"kb_used_data":56420,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94240016,"statfs":{"total":96636764160,"available":96501776384,"internally_reserved":0,"allocated":57774080,"data_stored":59183988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73999999999999999,"15min":0.751},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80600000000000005,"15min":0.79900000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.88100000000000001,"15min":0.91400000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":0.52700000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.0109999999999999,"15min":1.0389999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.95499999999999996},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.492,"5min":1.651,"15min":1.651},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.98299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.95999999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3049999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.107,"15min":1.131},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5840000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.98199999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":0.93600000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509474,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86116,"kb_used_data":59100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94285724,"statfs":{"total":96636764160,"available":96548581376,"internally_reserved":0,"allocated":60518400,"data_stored":64892073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.625,"15min":0.64400000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.995,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.036,"5min":1.214,"15min":1.214},"last":0.98399999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96899999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.749},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.407,"15min":1.407},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.121,"5min":1.415,"15min":1.415},"last":0.872},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.92200000000000004,"15min":0.94199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2270000000000001,"5min":1.448,"15min":1.448},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96599999999999997,"15min":0.98599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.96299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2010000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.052}]},{"osd":6,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.47399999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.353,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.878,"5min":1.0309999999999999,"15min":1.0569999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3460000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.083}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.1180000000000001}]}]},{"osd":0,"up_from":0,"seq":64424509474,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69020,"kb_used_data":37908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94302820,"statfs":{"total":96636764160,"available":96566087680,"internally_reserved":0,"allocated":38817792,"data_stored":44941130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77700000000000002,"15min":0.76400000000000001},"min":{"1min":0.47499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.70199999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.82199999999999995,"15min":0.79700000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.92900000000000005,"15min":0.92800000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":0.97199999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":0.94899999999999995,"15min":0.90100000000000002},"min":{"1min":0.53700000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":8.3499999999999996,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.206,"5min":0.98199999999999998,"15min":0.94399999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":8.2989999999999995,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.99399999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.95099999999999996,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.472,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.92300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":1.038}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.218,"5min":1.7,"15min":1.7},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.085,"15min":1.093},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0700000000000001,"15min":1.0720000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.091}]}]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104772,"kb_used_data":69556,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94267068,"statfs":{"total":96636764160,"available":96529477632,"internally_reserved":0,"allocated":71225344,"data_stored":78149784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.115,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":1.3,"15min":1.3},"last":0.86599999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.84799999999999998,"15min":0.81999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.90900000000000003,"15min":0.91600000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.399}]},{"osd":3,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.873,"15min":0.85299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.88900000000000001,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.248}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.5269999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.008,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.1020000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.206},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.3400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122236,"kb_used_data":64884,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94249604,"statfs":{"total":96636764160,"available":96511594496,"internally_reserved":0,"allocated":66441216,"data_stored":68714546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":1.1200000000000001,"15min":1.179},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0489999999999999,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":1.0680000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.085,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.218,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.1080000000000001,"15min":1.159},"min":{"1min":0.54900000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.151,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.026}]},{"osd":3,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4910000000000001,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.96999999999999997,"15min":0.97799999999999998},"min":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.585},"last":0.83899999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0169999999999999,"15min":1.0309999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.123},{"interface":"front","average":{"1min":1.006,"5min":1.357,"15min":1.415},"min":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3799999999999999,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.2090000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.012,"5min":1.0800000000000001,"15min":1.091},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4710000000000001,"5min":1.899,"15min":1.899},"last":1.1799999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.2450000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6295552,"data_stored":7615357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58417152,"data_stored":63600089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":56336384,"data_stored":58545765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":72056832,"data_stored":82069291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":69713920,"data_stored":77441026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":66441216,"data_stored":69531554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":37310464,"data_stored":44232514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":35127296,"data_stored":38868012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:09:31.035 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:09:31.035 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:09:31.035 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T09:09:31.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T09:09:31.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:31.038 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:31.034+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:31.138 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:31.134+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:31.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-13T09:09:31.209 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:31.210 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:31.220 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:09:31.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:31.244 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:31.242+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:31.339 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:31.334+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:31.366 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:31.366 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:31.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:31.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T09:09:31.484 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:31.482+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:31.501 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:31.501 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:31.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:31.541 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:31.538+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:31.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T09:09:31.642 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:31.636+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:31.642 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:31.643 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:31.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:31.749 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:31.744+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:31.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-13T09:09:31.831 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:31.831 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:31.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:31.863 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:31.862+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:31.945 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:31.940+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:31.982 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:31.982 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:31.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-13T09:09:31.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:32.045 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:32.042+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:32.146 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:32.142+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:32.170 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:32.170 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:32.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:32.251 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:32.240+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:32.347 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:32.346+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:32.355 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:32.355 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:32.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:32.449 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:32.444+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:32.541 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:32.542 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:32.550 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:32.546+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:32.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:32.652 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:32.644+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:32.728 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:32.728 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:32.742 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:32.752 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:32.744+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:32.853 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:32.850+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:32.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:32.870 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:32.870 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:32.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:32.953 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:32.952+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:33.018 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:33.018 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:33.036 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:33.036 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:33.050 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:33.051 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:33.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:33.053 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:33.054 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:33.052+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:33.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:33.070 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:33.070 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:33.091 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:33.091 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:33.111 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:33.112 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:33.132 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:33.133 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:33.153 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:33.154 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:33.154 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:33.148+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:33.194 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:33.194 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:33.195 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:33.195 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:33.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:33.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:09:33.255 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:33.248+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:33.356 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:33.354+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20616 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5324800~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.04334386399999, 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042961581999999998, 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20616 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338567+0000", 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338569+0000", 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338576+0000", 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338580+0000", 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.126e-06 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381334+0000", 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042754069999999998 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381418+0000", 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4303999999999998e-05 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381529+0000", 2024-09-13T09:09:33.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0001108 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.04332709099999, 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:09:33.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043289786, 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043270072, 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:09:33.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:09:33.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:33.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.04325374699999, 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:09:33.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043237881, 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:09:33.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.04322209599999, 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:09:33.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:33.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043203025, 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:09:33.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043186478, 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:09:33.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.426 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043158879, 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:09:33.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.04314163700001, 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:09:33.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.04311119400001, 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.436 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:09:33.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.04309227900001, 2024-09-13T09:09:33.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:09:33.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:09:33.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043074491, 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043049267, 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:09:33.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043031063, 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:09:33.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.043010745, 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:09:33.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.042992197, 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:09:33.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.468 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.042974923, 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:09:33.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 118.042957469, 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:09:33.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.478 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:33.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:33.479 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:33.472+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:33.580 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:33.572+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:33.649 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:33.649 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:33.663 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:33.707 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:33.706+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:33.781 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:33.776+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:33.788 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:33.788 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:33.788 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:33.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:33.887 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:33.886+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:33.974 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:09:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:33.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 149.83911573500001, 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:33.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:09:33.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:09:33.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:33.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 101.010833555, 2024-09-13T09:09:33.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:09:33.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:09:33.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.718393716999998, 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:09:33.986 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:09:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:09:33.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:09:33.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.718313918999996, 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:33.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:09:33.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:33.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.718064847999997, 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.997 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:33.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:33.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.717751707000005, 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:09:34.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.717298475000007, 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:34.010 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.717273961000004, 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:34.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:09:34.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.717008731999996, 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:09:34.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:09:34.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:34.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.716720882999994, 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:09:34.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:09:34.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.716259578999995, 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.024 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:09:34.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:09:34.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.777726229999999, 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:34.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:09:34.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:09:34.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.766732910000002, 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:09:34.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:09:34.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.762751012000003, 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.042 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:09:34.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:34.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.760123018000002, 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:09:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:09:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:09:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.758570233999997, 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:09:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:09:34.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.757316918000001, 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:09:34.063 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:09:34.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.756214884999999, 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:09:34.070 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:09:34.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.755004307999997, 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:09:34.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:09:34.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3:f1cafc87:::rbd_data.1124651f5f97.000000000000004a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 557056~22319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.847308893000001, 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97276468599999999, 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:34.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:09:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092738+0000", 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97003675 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:34.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092877+0000", 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139245 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093029+0000", 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015224999999999999 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:34.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093847+0000", 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081788800000000001 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095250+0000", 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014028529999999999 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095282+0000", 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1961000000000001e-05 2024-09-13T09:09:34.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095303+0000", 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:34.091 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:34.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:34.093 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:34.088+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:34.194 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:34.190+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:34.226 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:34.227 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:34.227 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:34.227 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:34.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:34.242 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:09:34.294 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:34.288+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:34.386 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:34.387 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:34.387 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:34.387 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:34.387 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:34.395 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:34.394+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:34.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:34.497 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:34.494+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:34.576 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:34.576 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:34.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:34.576 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:34.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.576 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.548203547, 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.577 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.581 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.548156214000002, 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.582 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:09:34.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:09:34.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.586 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:09:34.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:09:34.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:09:34.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:09:34.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:34.588 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.548098678000002, 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:09:34.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:34.593 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.548047191999999, 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.595 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:34.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.595 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:34.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:09:34.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:09:34.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:09:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.548000250000001, 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:09:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:09:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.547952856999999, 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.547905581999999, 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:09:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:09:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.547859381999999, 2024-09-13T09:09:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:09:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.547795377, 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:09:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:09:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.545324288000003, 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:09:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:09:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.545268566000004, 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:09:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.545218663, 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.545171607, 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:09:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.545123124, 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:09:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.545057858999996, 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:09:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:09:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.545010942999994, 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:09:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:09:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.544923495000006, 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:09:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:09:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.543461645999997, 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:09:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:09:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.543383559999995, 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:09:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.543331471000002, 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:09:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:34.675 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:34.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:34.677 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:34.672+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:34.778 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:34.772+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:34.852 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:34.852 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:34.852 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:34.852 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:34.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:34.879 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:34.874+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:34.979 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:34.972+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:35.036 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:35.036 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:35.036 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:35.036 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:35.037 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:35.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:35.097 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:35.092+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:35.172 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:35.180 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:35.176+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.598384052, 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:09:35.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:35.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.224 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.598360722, 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:09:35.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:35.229 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.59833770500001, 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.233 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.598314575, 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:35.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.595697812, 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.238 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:09:35.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.595673726, 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:09:35.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.59564707299999, 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.247 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:09:35.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.595623433, 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.252 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:09:35.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:09:35.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.256 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.59180789200001, 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.257 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:09:35.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.59178434899999, 2024-09-13T09:09:35.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.265 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.59176105900001, 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:35.270 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.591737546, 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:09:35.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:09:35.274 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.59167909, 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.275 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:35.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:09:35.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 120.58764815399999, 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.280 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 102.267749857, 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:09:35.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:09:35.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:09:35.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:35.289 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 102.265378078, 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:35.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:09:35.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 102.26484416300001, 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:09:35.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:09:35.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 102.26144364, 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:09:35.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:09:35.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 102.260443833, 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:35.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:09:35.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:09:35.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 102.257273555, 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:35.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:09:35.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.313 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.314 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.314 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:35.314 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:35.314 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:35.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:35.316 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:35.308+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:35.339 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:35.339 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:35.356 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:35.373 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:35.373 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:35.390 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:35.391 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:35.411 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:35.411 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:35.415 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:35.413+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:35.429 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:35.429 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:35.447 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:35.447 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:35.465 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:35.465 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:35.496 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:35.497 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:35.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:35.561 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:35.556+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:35.617 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:35.612+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:35.634 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:35.634 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:35.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:35.741 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:35.736+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:35.819 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:35.812+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:35.824 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:35.824 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:35.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:35.929 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:35.924+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:36.008 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:36.009 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:36.020 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:36.012+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:36.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:36.121 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:36.117+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:36.192 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:36.192 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:36.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:36.253 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:36.248+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:36.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:09:36.327 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:36.325+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:36.346 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:36.346 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:36.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:36.424 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:36.416+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:36.494 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:36.494 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:36.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:36.524 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:36.520+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:36.625 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:36.624+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:36.626 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:36.626 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:36.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:36.726 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:36.720+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:36.769 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:36.769 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:36.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:36.827 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:36.820+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:36.903 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:36.903 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:36.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:36.929 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:36.924+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:37.030 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:37.025+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:37.037 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:37.037 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:37.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:37.139 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:37.137+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:37.149 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1] out_osds: [7, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:37.149 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:09:37.149 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T09:09:37.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T09:09:37.224 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:37.224 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:37.232 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:37.224+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:37.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:37.279 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:09:37.332 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:37.328+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:37.375 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:37.375 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:37.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:37.435 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:37.433+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:37.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 140, 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "fe16bf6d-4166-4417-80fd-1123617a33d3", 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-09-13T09:06:44.149006+0000", 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-09-13T09:09:37.265643+0000", 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-09-13T09:06:55.611487+0000", 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-09-13T09:08:59.952135+0000", 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5801984, 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-09-13T09:09:37.494 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T09:09:37.495 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "reef", 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:06:55.131330+0000", 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-09-13T09:09:37.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-09-13T09:09:37.497 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:37.498 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:37.499 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:37.500 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-09-13T09:09:37.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-09-13T09:09:37.502 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:02.743102+0000", 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-09-13T09:09:37.503 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:37.504 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 3, 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:37.505 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:37.506 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 4.9699997901916504, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 4.9699997901916504, 2024-09-13T09:09:37.507 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.62999999523162842, 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 3.130000114440918, 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 3.130000114440918, 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-09-13T09:07:12.517158+0000", 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-09-13T09:09:37.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-09-13T09:09:37.509 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "140", 2024-09-13T09:09:37.510 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 60, 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 140, 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-09-13T09:09:37.511 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-09-13T09:09:37.512 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.513 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.75, 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.309999942779541, 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.309999942779541, 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.99993515014648438, 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:09:37.514 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "33c6fc7d-9828-4500-b103-b96d1cd7f420", 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 107, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:37.515 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:37.516 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.518 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3291756747 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6818/3291756747", 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6822/3291756747", 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6830/3291756747", 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6826/3291756747", 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.520 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "869ea3c5-8e6e-4b49-b70e-083f242a7aae", 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.999481201171875, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 126, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:37.521 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:37.522 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.523 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 533520566 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.525 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/533520566", 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/533520566", 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/533520566", 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/533520566", 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e144d496-76ee-46e0-bdc6-135c77128535", 2024-09-13T09:09:37.526 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 125, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.527 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:37.530 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2253604482 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6819/2253604482", 2024-09-13T09:09:37.531 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6823/2253604482", 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/2253604482", 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6827/2253604482", 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "696b9414-9c19-4984-a8ff-14548e46b357", 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:37.532 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 118, 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:37.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:37.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3367214693 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/3367214693", 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/3367214693", 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/3367214693", 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/3367214693", 2024-09-13T09:09:37.537 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "ac58040d-9b54-42ca-b143-1816abb54816", 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:37.538 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6800", 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6801", 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6802", 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6803", 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6806", 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6807", 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6804", 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6805", 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3778074750 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6801/3778074750", 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6803/3778074750", 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6807/3778074750", 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6805/3778074750", 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:37.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2", 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:37.544 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6808", 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6809", 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6811", 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6813", 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6819", 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6824", 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.547 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6815", 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6817", 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1926603087 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6809/1926603087", 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6813/1926603087", 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6824/1926603087", 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6817/1926603087", 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:09:37.549 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d", 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 14, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 103, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.550 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6820", 2024-09-13T09:09:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.551 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6821", 2024-09-13T09:09:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6823", 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6825", 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:37.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6830", 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6831", 2024-09-13T09:09:37.554 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6827", 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.555 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6828", 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 766305500 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6821/766305500", 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6825/766305500", 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6831/766305500", 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6828/766305500", 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:37.556 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "f36c332c-9497-47e8-b943-f6b6dcaebeb8", 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 0, 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0, 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-09-13T09:09:37.557 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 15, 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 86, 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6810", 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6812", 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6814", 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6816", 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6826", 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:37.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6829", 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6818", 2024-09-13T09:09:37.561 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.60:6822", 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 1032956039 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.60:6812/1032956039", 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.60:6816/1032956039", 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.60:6829/1032956039", 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.60:6822/1032956039", 2024-09-13T09:09:37.562 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:37.563 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 16238, 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.183436+0000", 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.264080+0000", 2024-09-13T09:09:37.564 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.253587+0000", 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.244429+0000", 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:37.566 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.369458+0000", 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:37.567 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.158089+0000", 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.217295+0000", 2024-09-13T09:09:37.568 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540138322906710015, 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 65536, 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-09-13T09:06:52.332254+0000", 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.569 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.a", 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: 4, 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: 2 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [ 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.f", 2024-09-13T09:09:37.570 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 1, 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 3 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.14", 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 3, 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 1 2024-09-13T09:09:37.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-09-13T09:09:37.572 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 43, 2024-09-13T09:09:37.573 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 9 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 43, 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-09-13T09:09:37.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-09-13T09:09:37.575 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-09-13T09:09:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-09-13T09:09:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-09-13T09:09:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-09-13T09:09:37.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:37.576 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:37.578 INFO:tasks.thrashosds.thrasher:j is {'epoch': 140, 'fsid': 'fe16bf6d-4166-4417-80fd-1123617a33d3', 'created': '2024-09-13T09:06:44.149006+0000', 'modified': '2024-09-13T09:09:37.265643+0000', 'last_up_change': '2024-09-13T09:06:55.611487+0000', 'last_in_change': '2024-09-13T09:08:59.952135+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T09:06:55.131330+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-13T09:07:02.743102+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.96999979019165, 'score_stable': 4.96999979019165, 'optimal_score': 0.6299999952316284, 'raw_score_acting': 3.130000114440918, 'raw_score_stable': 3.130000114440918, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T09:07:12.517158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 60, 'snap_epoch': 140, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 0.75, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.9999351501464844, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '33c6fc7d-9828-4500-b103-b96d1cd7f420', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 107, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6818', 'nonce': 3291756747}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6822', 'nonce': 3291756747}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6830', 'nonce': 3291756747}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 3291756747}, {'type': 'v1', 'addr': '172.21.15.31:6826', 'nonce': 3291756747}]}, 'public_addr': '172.21.15.31:6818/3291756747', 'cluster_addr': '172.21.15.31:6822/3291756747', 'heartbeat_back_addr': '172.21.15.31:6830/3291756747', 'heartbeat_front_addr': '172.21.15.31:6826/3291756747', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '869ea3c5-8e6e-4b49-b70e-083f242a7aae', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.999481201171875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 126, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 533520566}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 533520566}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 533520566}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 533520566}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 533520566}]}, 'public_addr': '172.21.15.31:6801/533520566', 'cluster_addr': '172.21.15.31:6803/533520566', 'heartbeat_back_addr': '172.21.15.31:6807/533520566', 'heartbeat_front_addr': '172.21.15.31:6805/533520566', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'e144d496-76ee-46e0-bdc6-135c77128535', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6817', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2253604482}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6821', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2253604482}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6829', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 2253604482}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6825', 'nonce': 2253604482}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 2253604482}]}, 'public_addr': '172.21.15.31:6819/2253604482', 'cluster_addr': '172.21.15.31:6823/2253604482', 'heartbeat_back_addr': '172.21.15.31:6831/2253604482', 'heartbeat_front_addr': '172.21.15.31:6827/2253604482', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '696b9414-9c19-4984-a8ff-14548e46b357', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 118, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 3367214693}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 3367214693}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 3367214693}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 3367214693}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 3367214693}]}, 'public_addr': '172.21.15.31:6809/3367214693', 'cluster_addr': '172.21.15.31:6811/3367214693', 'heartbeat_back_addr': '172.21.15.31:6815/3367214693', 'heartbeat_front_addr': '172.21.15.31:6813/3367214693', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ac58040d-9b54-42ca-b143-1816abb54816', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6800', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6801', 'nonce': 3778074750}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 3778074750}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 3778074750}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 3778074750}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 3778074750}]}, 'public_addr': '172.21.15.60:6801/3778074750', 'cluster_addr': '172.21.15.60:6803/3778074750', 'heartbeat_back_addr': '172.21.15.60:6807/3778074750', 'heartbeat_front_addr': '172.21.15.60:6805/3778074750', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1926603087}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6811', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 1926603087}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6819', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6824', 'nonce': 1926603087}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6815', 'nonce': 1926603087}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 1926603087}]}, 'public_addr': '172.21.15.60:6809/1926603087', 'cluster_addr': '172.21.15.60:6813/1926603087', 'heartbeat_back_addr': '172.21.15.60:6824/1926603087', 'heartbeat_front_addr': '172.21.15.60:6817/1926603087', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'd1c0a8d6-9346-4d44-9fa2-e15629b7ba4d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 766305500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6823', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 766305500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 766305500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6827', 'nonce': 766305500}, {'type': 'v1', 'addr': '172.21.15.60:6828', 'nonce': 766305500}]}, 'public_addr': '172.21.15.60:6821/766305500', 'cluster_addr': '172.21.15.60:6825/766305500', 'heartbeat_back_addr': '172.21.15.60:6831/766305500', 'heartbeat_front_addr': '172.21.15.60:6828/766305500', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'f36c332c-9497-47e8-b943-f6b6dcaebeb8', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6812', 'nonce': 1032956039}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6816', 'nonce': 1032956039}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 1032956039}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 1032956039}, {'type': 'v1', 'addr': '172.21.15.60:6822', 'nonce': 1032956039}]}, 'public_addr': '172.21.15.60:6812/1032956039', 'cluster_addr': '172.21.15.60:6816/1032956039', 'heartbeat_back_addr': '172.21.15.60:6829/1032956039', 'heartbeat_front_addr': '172.21.15.60:6822/1032956039', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 16238, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.183436+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.264080+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.253587+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.244429+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.369458+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.158089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.217295+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-13T09:06:52.332254+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.a', 'osds': [4, 2]}], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 1, 'to': 3}]}, {'pgid': '3.14', 'mappings': [{'from': 3, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 43, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 43, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T09:09:37.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:09:37.579 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:37.579 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:37.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:37.580 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:37.573+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:37.620 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:37.620 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:37.638 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:37.638 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:37.679 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:37.677+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:37.732 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:37.732 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:37.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:37.796 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:37.793+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:37.857 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:37.858 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:37.877 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:37.877 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:37.880 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:37.877+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:37.898 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:37.898 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:37.921 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:09:37.922 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:37.922 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:09:37.923 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:37.923 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:37.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:37.942 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-13T09:09:37.273268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1511286462,"num_objects":1564,"num_object_clones":331,"num_object_copies":3128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1564,"num_whiteouts":617,"num_read":199731,"num_read_kb":681004,"num_write":23131,"num_write_kb":339971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":175529217,"num_keys_recovered":405,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18641,"ondisk_log_size":18641,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":644464,"kb_used_data":342864,"kb_used_omap":9,"kb_used_meta":301430,"kb_avail":565586576,"statfs":{"total":579820584960,"available":579160653824,"internally_reserved":0,"allocated":351092736,"data_stored":378164769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9561,"internal_metadata":308664999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-42347257,"num_objects":-203,"num_object_clones":-59,"num_object_copies":-406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-203,"num_whiteouts":-45,"num_read":1027,"num_read_kb":1018,"num_write":261,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040480"},"pg_stats":[{"pgid":"3.1f","version":"135'499","reported_seq":5404,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-13T09:09:33.239360+0000","last_change":"2024-09-13T09:09:33.239844+0000","last_active":"2024-09-13T09:09:33.239360+0000","last_peered":"2024-09-13T09:09:33.239360+0000","last_clean":"2024-09-13T09:09:33.239360+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:09:33.239360+0000","last_undegraded":"2024-09-13T09:09:33.239360+0000","last_fullsized":"2024-09-13T09:09:33.239360+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003247144,"stat_sum":{"num_bytes":37991697,"num_objects":60,"num_object_clones":15,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":27,"num_read":5902,"num_read_kb":20167,"num_write":634,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"5"}]},{"pgid":"3.1e","version":"136'1091","reported_seq":4495,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-13T09:09:33.253624+0000","last_change":"2024-09-13T09:09:33.239418+0000","last_active":"2024-09-13T09:09:33.253624+0000","last_peered":"2024-09-13T09:09:33.253624+0000","last_clean":"2024-09-13T09:09:33.253624+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:09:33.253624+0000","last_undegraded":"2024-09-13T09:09:33.253624+0000","last_fullsized":"2024-09-13T09:09:33.253624+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002194882,"stat_sum":{"num_bytes":46815109,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":8595,"num_read_kb":18612,"num_write":1470,"num_write_kb":6006,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"5"}]},{"pgid":"3.1d","version":"137'1353","reported_seq":5124,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.415529+0000","last_change":"2024-09-13T09:09:35.263399+0000","last_active":"2024-09-13T09:09:35.415529+0000","last_peered":"2024-09-13T09:09:35.415529+0000","last_clean":"2024-09-13T09:09:35.415529+0000","last_became_active":"2024-09-13T09:08:38.208191+0000","last_became_peered":"2024-09-13T09:08:38.208191+0000","last_unstale":"2024-09-13T09:09:35.415529+0000","last_undegraded":"2024-09-13T09:09:35.415529+0000","last_fullsized":"2024-09-13T09:09:35.415529+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:04.686300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002700441,"stat_sum":{"num_bytes":53392166,"num_objects":60,"num_object_clones":15,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":30,"num_read":8702,"num_read_kb":34125,"num_write":2206,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.1c","version":"138'616","reported_seq":3543,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.333242+0000","last_change":"2024-09-13T09:09:35.290535+0000","last_active":"2024-09-13T09:09:35.333242+0000","last_peered":"2024-09-13T09:09:35.333242+0000","last_clean":"2024-09-13T09:09:35.333242+0000","last_became_active":"2024-09-13T09:08:38.209451+0000","last_became_peered":"2024-09-13T09:08:38.209451+0000","last_unstale":"2024-09-13T09:09:35.333242+0000","last_undegraded":"2024-09-13T09:09:35.333242+0000","last_fullsized":"2024-09-13T09:09:35.333242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:59.672566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014153849999999999,"stat_sum":{"num_bytes":77031665,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":5292,"num_read_kb":35209,"num_write":842,"num_write_kb":15628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.1b","version":"138'612","reported_seq":21590,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.400426+0000","last_change":"2024-09-13T09:09:35.277765+0000","last_active":"2024-09-13T09:09:35.400426+0000","last_peered":"2024-09-13T09:09:35.400426+0000","last_clean":"2024-09-13T09:09:35.400426+0000","last_became_active":"2024-09-13T09:08:38.210333+0000","last_became_peered":"2024-09-13T09:08:38.210333+0000","last_unstale":"2024-09-13T09:09:35.400426+0000","last_undegraded":"2024-09-13T09:09:35.400426+0000","last_fullsized":"2024-09-13T09:09:35.400426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:53.493070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014348780000000001,"stat_sum":{"num_bytes":91395080,"num_objects":52,"num_object_clones":11,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":19,"num_read":23015,"num_read_kb":47613,"num_write":705,"num_write_kb":18551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.1a","version":"136'583","reported_seq":5629,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.420558+0000","last_change":"2024-09-13T09:09:35.289308+0000","last_active":"2024-09-13T09:09:35.420558+0000","last_peered":"2024-09-13T09:09:35.420558+0000","last_clean":"2024-09-13T09:09:35.420558+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:09:35.420558+0000","last_undegraded":"2024-09-13T09:09:35.420558+0000","last_fullsized":"2024-09-13T09:09:35.420558+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078550199999999999,"stat_sum":{"num_bytes":60629886,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":24,"num_read":7493,"num_read_kb":25002,"num_write":763,"num_write_kb":19650,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.19","version":"137'330","reported_seq":1737,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.278235+0000","last_change":"2024-09-13T09:09:35.278501+0000","last_active":"2024-09-13T09:09:35.278235+0000","last_peered":"2024-09-13T09:09:35.278235+0000","last_clean":"2024-09-13T09:09:35.278235+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:09:35.278235+0000","last_undegraded":"2024-09-13T09:09:35.278235+0000","last_fullsized":"2024-09-13T09:09:35.278235+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016557290000000001,"stat_sum":{"num_bytes":14411960,"num_objects":48,"num_object_clones":7,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":1638,"num_read_kb":7983,"num_write":310,"num_write_kb":7298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.18","version":"135'592","reported_seq":2953,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-13T09:09:33.237036+0000","last_change":"2024-09-13T09:09:33.237540+0000","last_active":"2024-09-13T09:09:33.237036+0000","last_peered":"2024-09-13T09:09:33.237036+0000","last_clean":"2024-09-13T09:09:33.237036+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:09:33.237036+0000","last_undegraded":"2024-09-13T09:09:33.237036+0000","last_fullsized":"2024-09-13T09:09:33.237036+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080901300000000003,"stat_sum":{"num_bytes":45848292,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":22,"num_read":4164,"num_read_kb":14248,"num_write":678,"num_write_kb":10427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"5"}]},{"pgid":"3.17","version":"135'590","reported_seq":5451,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-13T09:09:33.238314+0000","last_change":"2024-09-13T09:09:33.240093+0000","last_active":"2024-09-13T09:09:33.238314+0000","last_peered":"2024-09-13T09:09:33.238314+0000","last_clean":"2024-09-13T09:09:33.238314+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:09:33.238314+0000","last_undegraded":"2024-09-13T09:09:33.238314+0000","last_fullsized":"2024-09-13T09:09:33.238314+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024641670000000002,"stat_sum":{"num_bytes":53355709,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":6878,"num_read_kb":30983,"num_write":589,"num_write_kb":18259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"5"}]},{"pgid":"3.16","version":"135'655","reported_seq":1817,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.271812+0000","last_change":"2024-09-13T09:09:35.272288+0000","last_active":"2024-09-13T09:09:35.271812+0000","last_peered":"2024-09-13T09:09:35.271812+0000","last_clean":"2024-09-13T09:09:35.271812+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:09:35.271812+0000","last_undegraded":"2024-09-13T09:09:35.271812+0000","last_fullsized":"2024-09-13T09:09:35.271812+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060727700000000003,"stat_sum":{"num_bytes":35472392,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":19,"num_read":2872,"num_read_kb":14168,"num_write":926,"num_write_kb":9801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.15","version":"137'620","reported_seq":3878,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.388199+0000","last_change":"2024-09-13T09:09:34.254395+0000","last_active":"2024-09-13T09:09:34.388199+0000","last_peered":"2024-09-13T09:09:34.388199+0000","last_clean":"2024-09-13T09:09:34.388199+0000","last_became_active":"2024-09-13T09:08:38.221484+0000","last_became_peered":"2024-09-13T09:08:38.221484+0000","last_unstale":"2024-09-13T09:09:34.388199+0000","last_undegraded":"2024-09-13T09:09:34.388199+0000","last_fullsized":"2024-09-13T09:09:34.388199+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:04.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032131109999999998,"stat_sum":{"num_bytes":92172088,"num_objects":53,"num_object_clones":16,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":17,"num_read":4827,"num_read_kb":24392,"num_write":622,"num_write_kb":16150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"6"}]},{"pgid":"3.14","version":"135'601","reported_seq":18806,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.289420+0000","last_change":"2024-09-13T09:09:35.289786+0000","last_active":"2024-09-13T09:09:35.289420+0000","last_peered":"2024-09-13T09:09:35.289420+0000","last_clean":"2024-09-13T09:09:35.289420+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:09:35.289420+0000","last_undegraded":"2024-09-13T09:09:35.289420+0000","last_fullsized":"2024-09-13T09:09:35.289420+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00104101,"stat_sum":{"num_bytes":46468143,"num_objects":54,"num_object_clones":13,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":17,"num_read":20437,"num_read_kb":33589,"num_write":550,"num_write_kb":10175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.257875+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:09:34.257875+0000","last_peered":"2024-09-13T09:09:34.257875+0000","last_clean":"2024-09-13T09:09:34.257875+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:09:34.257875+0000","last_undegraded":"2024-09-13T09:09:34.257875+0000","last_fullsized":"2024-09-13T09:09:34.257875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"138'582","reported_seq":5315,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.439476+0000","last_change":"2024-09-13T09:09:35.278135+0000","last_active":"2024-09-13T09:09:35.439476+0000","last_peered":"2024-09-13T09:09:35.439476+0000","last_clean":"2024-09-13T09:09:35.439476+0000","last_became_active":"2024-09-13T09:08:57.961197+0000","last_became_peered":"2024-09-13T09:08:57.961197+0000","last_unstale":"2024-09-13T09:09:35.439476+0000","last_undegraded":"2024-09-13T09:09:35.439476+0000","last_fullsized":"2024-09-13T09:09:35.439476+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:00.864515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016497389999999999,"stat_sum":{"num_bytes":49677175,"num_objects":40,"num_object_clones":7,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":13,"num_read":5936,"num_read_kb":21747,"num_write":727,"num_write_kb":6954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":6759140,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.243247+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:09:34.243247+0000","last_peered":"2024-09-13T09:09:34.243247+0000","last_clean":"2024-09-13T09:09:34.243247+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:09:34.243247+0000","last_undegraded":"2024-09-13T09:09:34.243247+0000","last_fullsized":"2024-09-13T09:09:34.243247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"137'388","reported_seq":2168,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.277817+0000","last_change":"2024-09-13T09:09:35.278134+0000","last_active":"2024-09-13T09:09:35.277817+0000","last_peered":"2024-09-13T09:09:35.277817+0000","last_clean":"2024-09-13T09:09:35.277817+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:09:35.277817+0000","last_undegraded":"2024-09-13T09:09:35.277817+0000","last_fullsized":"2024-09-13T09:09:35.277817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016403539999999999,"stat_sum":{"num_bytes":23933200,"num_objects":38,"num_object_clones":5,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":2515,"num_read_kb":14465,"num_write":358,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":134,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.256780+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:09:34.256780+0000","last_peered":"2024-09-13T09:09:34.256780+0000","last_clean":"2024-09-13T09:09:34.256780+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:09:34.256780+0000","last_undegraded":"2024-09-13T09:09:34.256780+0000","last_fullsized":"2024-09-13T09:09:34.256780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"136'578","reported_seq":5951,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.250085+0000","last_change":"2024-09-13T09:09:34.250259+0000","last_active":"2024-09-13T09:09:34.250085+0000","last_peered":"2024-09-13T09:09:34.250085+0000","last_clean":"2024-09-13T09:09:34.250085+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:09:34.250085+0000","last_undegraded":"2024-09-13T09:09:34.250085+0000","last_fullsized":"2024-09-13T09:09:34.250085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011591209999999999,"stat_sum":{"num_bytes":48989781,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":14,"num_read":6776,"num_read_kb":25994,"num_write":783,"num_write_kb":6960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-13T09:09:32.229437+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:09:32.229437+0000","last_peered":"2024-09-13T09:09:32.229437+0000","last_clean":"2024-09-13T09:09:32.229437+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:09:32.229437+0000","last_undegraded":"2024-09-13T09:09:32.229437+0000","last_fullsized":"2024-09-13T09:09:32.229437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"136'626","reported_seq":3520,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.272351+0000","last_change":"2024-09-13T09:09:35.272843+0000","last_active":"2024-09-13T09:09:35.272351+0000","last_peered":"2024-09-13T09:09:35.272351+0000","last_clean":"2024-09-13T09:09:35.272351+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:09:35.272351+0000","last_undegraded":"2024-09-13T09:09:35.272351+0000","last_fullsized":"2024-09-13T09:09:35.272351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":626,"log_dups_size":0,"ondisk_log_size":626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000890623,"stat_sum":{"num_bytes":40574787,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":15,"num_read":4759,"num_read_kb":19626,"num_write":819,"num_write_kb":10459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.3","version":"138'437","reported_seq":3022,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.412669+0000","last_change":"2024-09-13T09:09:35.272228+0000","last_active":"2024-09-13T09:09:35.412669+0000","last_peered":"2024-09-13T09:09:35.412669+0000","last_clean":"2024-09-13T09:09:35.412669+0000","last_became_active":"2024-09-13T09:08:38.209914+0000","last_became_peered":"2024-09-13T09:08:38.209914+0000","last_unstale":"2024-09-13T09:09:35.412669+0000","last_undegraded":"2024-09-13T09:09:35.412669+0000","last_fullsized":"2024-09-13T09:09:35.412669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:12:19.947594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051478199999999998,"stat_sum":{"num_bytes":78970947,"num_objects":58,"num_object_clones":13,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":26,"num_read":3091,"num_read_kb":18019,"num_write":440,"num_write_kb":6087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":918446,"num_keys_recovered":11,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"2.2","version":"21'2","reported_seq":136,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.256154+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:09:34.256154+0000","last_peered":"2024-09-13T09:09:34.256154+0000","last_clean":"2024-09-13T09:09:34.256154+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:09:34.256154+0000","last_undegraded":"2024-09-13T09:09:34.256154+0000","last_fullsized":"2024-09-13T09:09:34.256154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"135'404","reported_seq":3011,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-09-13T09:09:33.236296+0000","last_change":"2024-09-13T09:09:33.237210+0000","last_active":"2024-09-13T09:09:33.236296+0000","last_peered":"2024-09-13T09:09:33.236296+0000","last_clean":"2024-09-13T09:09:33.236296+0000","last_became_active":"2024-09-13T09:08:57.957875+0000","last_became_peered":"2024-09-13T09:08:57.957875+0000","last_unstale":"2024-09-13T09:09:33.236296+0000","last_undegraded":"2024-09-13T09:09:33.236296+0000","last_fullsized":"2024-09-13T09:09:33.236296+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:45:37.313991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001211204,"stat_sum":{"num_bytes":25007723,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":3178,"num_read_kb":18973,"num_write":438,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":3111448,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2f","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-09-13T09:09:32.228878+0000","last_change":"2024-09-13T09:08:54.932713+0000","last_active":"2024-09-13T09:09:32.228878+0000","last_peered":"2024-09-13T09:09:32.228878+0000","last_clean":"2024-09-13T09:09:32.228878+0000","last_became_active":"2024-09-13T09:08:54.931779+0000","last_became_peered":"2024-09-13T09:08:54.931779+0000","last_unstale":"2024-09-13T09:09:32.228878+0000","last_undegraded":"2024-09-13T09:09:32.228878+0000","last_fullsized":"2024-09-13T09:09:32.228878+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:42:37.917578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"135'360","reported_seq":1967,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.249201+0000","last_change":"2024-09-13T09:09:34.249500+0000","last_active":"2024-09-13T09:09:34.249201+0000","last_peered":"2024-09-13T09:09:34.249201+0000","last_clean":"2024-09-13T09:09:34.249201+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:09:34.249201+0000","last_undegraded":"2024-09-13T09:09:34.249201+0000","last_fullsized":"2024-09-13T09:09:34.249201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070441999999999998,"stat_sum":{"num_bytes":15265327,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":19,"num_read":1866,"num_read_kb":14725,"num_write":372,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.255995+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:09:34.255995+0000","last_peered":"2024-09-13T09:09:34.255995+0000","last_clean":"2024-09-13T09:09:34.255995+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:09:34.255995+0000","last_undegraded":"2024-09-13T09:09:34.255995+0000","last_fullsized":"2024-09-13T09:09:34.255995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"135'349","reported_seq":2203,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.278182+0000","last_change":"2024-09-13T09:09:35.278365+0000","last_active":"2024-09-13T09:09:35.278182+0000","last_peered":"2024-09-13T09:09:35.278182+0000","last_clean":"2024-09-13T09:09:35.278182+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:09:35.278182+0000","last_undegraded":"2024-09-13T09:09:35.278182+0000","last_fullsized":"2024-09-13T09:09:35.278182+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017604649999999999,"stat_sum":{"num_bytes":16147545,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":15,"num_read":2335,"num_read_kb":12061,"num_write":384,"num_write_kb":3722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"2.3","version":"19'1","reported_seq":135,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.256634+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:09:34.256634+0000","last_peered":"2024-09-13T09:09:34.256634+0000","last_clean":"2024-09-13T09:09:34.256634+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:09:34.256634+0000","last_undegraded":"2024-09-13T09:09:34.256634+0000","last_fullsized":"2024-09-13T09:09:34.256634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":263,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.245946+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:09:34.245946+0000","last_peered":"2024-09-13T09:09:34.245946+0000","last_clean":"2024-09-13T09:09:34.245946+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:09:34.245946+0000","last_undegraded":"2024-09-13T09:09:34.245946+0000","last_fullsized":"2024-09-13T09:09:34.245946+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"137'832","reported_seq":5543,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.276572+0000","last_change":"2024-09-13T09:09:35.276769+0000","last_active":"2024-09-13T09:09:35.276572+0000","last_peered":"2024-09-13T09:09:35.276572+0000","last_clean":"2024-09-13T09:09:35.276572+0000","last_became_active":"2024-09-13T09:08:57.964033+0000","last_became_peered":"2024-09-13T09:08:57.964033+0000","last_unstale":"2024-09-13T09:09:35.276572+0000","last_undegraded":"2024-09-13T09:09:35.276572+0000","last_fullsized":"2024-09-13T09:09:35.276572+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:16:54.967175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051220900000000001,"stat_sum":{"num_bytes":56458063,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":6812,"num_read_kb":23174,"num_write":1246,"num_write_kb":11880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":4982043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.9","version":"132'506","reported_seq":3121,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.261591+0000","last_change":"2024-09-13T09:09:35.262048+0000","last_active":"2024-09-13T09:09:35.261591+0000","last_peered":"2024-09-13T09:09:35.261591+0000","last_clean":"2024-09-13T09:09:35.261591+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:09:35.261591+0000","last_undegraded":"2024-09-13T09:09:35.261591+0000","last_fullsized":"2024-09-13T09:09:35.261591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014925050000000001,"stat_sum":{"num_bytes":47878367,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":4352,"num_read_kb":16412,"num_write":620,"num_write_kb":9237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.a","version":"138'510","reported_seq":2675,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.277755+0000","last_change":"2024-09-13T09:09:35.277755+0000","last_active":"2024-09-13T09:09:35.277755+0000","last_peered":"2024-09-13T09:09:35.277755+0000","last_clean":"2024-09-13T09:09:35.277755+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:09:35.277755+0000","last_undegraded":"2024-09-13T09:09:35.277755+0000","last_fullsized":"2024-09-13T09:09:35.277755+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014769220000000001,"stat_sum":{"num_bytes":82520370,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":3657,"num_read_kb":22332,"num_write":495,"num_write_kb":17987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.b","version":"136'360","reported_seq":3441,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.249374+0000","last_change":"2024-09-13T09:09:34.249904+0000","last_active":"2024-09-13T09:09:34.249374+0000","last_peered":"2024-09-13T09:09:34.249374+0000","last_clean":"2024-09-13T09:09:34.249374+0000","last_became_active":"2024-09-13T09:08:57.961707+0000","last_became_peered":"2024-09-13T09:08:57.961707+0000","last_unstale":"2024-09-13T09:09:34.249374+0000","last_undegraded":"2024-09-13T09:09:34.249374+0000","last_fullsized":"2024-09-13T09:09:34.249374+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:40:58.156711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093587900000000003,"stat_sum":{"num_bytes":37304628,"num_objects":51,"num_object_clones":12,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":17,"num_read":3420,"num_read_kb":33772,"num_write":383,"num_write_kb":17801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":8453655,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"6"}]},{"pgid":"3.c","version":"135'298","reported_seq":16788,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.262460+0000","last_change":"2024-09-13T09:09:35.262954+0000","last_active":"2024-09-13T09:09:35.262460+0000","last_peered":"2024-09-13T09:09:35.262460+0000","last_clean":"2024-09-13T09:09:35.262460+0000","last_became_active":"2024-09-13T09:08:57.959990+0000","last_became_peered":"2024-09-13T09:08:57.959990+0000","last_unstale":"2024-09-13T09:09:35.262460+0000","last_undegraded":"2024-09-13T09:09:35.262460+0000","last_fullsized":"2024-09-13T09:09:35.262460+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:35:21.370554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021235859999999998,"stat_sum":{"num_bytes":49639484,"num_objects":50,"num_object_clones":11,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":16418,"num_read_kb":27782,"num_write":327,"num_write_kb":15667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":26451574,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.d","version":"137'358","reported_seq":2751,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.263462+0000","last_change":"2024-09-13T09:09:35.263713+0000","last_active":"2024-09-13T09:09:35.263462+0000","last_peered":"2024-09-13T09:09:35.263462+0000","last_clean":"2024-09-13T09:09:35.263462+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:09:35.263462+0000","last_undegraded":"2024-09-13T09:09:35.263462+0000","last_fullsized":"2024-09-13T09:09:35.263462+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026402700000000001,"stat_sum":{"num_bytes":31542849,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":2849,"num_read_kb":9608,"num_write":329,"num_write_kb":5226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.e","version":"137'761","reported_seq":6588,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.272288+0000","last_change":"2024-09-13T09:09:35.272772+0000","last_active":"2024-09-13T09:09:35.272288+0000","last_peered":"2024-09-13T09:09:35.272288+0000","last_clean":"2024-09-13T09:09:35.272288+0000","last_became_active":"2024-09-13T09:08:38.205194+0000","last_became_peered":"2024-09-13T09:08:38.205194+0000","last_unstale":"2024-09-13T09:09:35.272288+0000","last_undegraded":"2024-09-13T09:09:35.272288+0000","last_fullsized":"2024-09-13T09:09:35.272288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:14.407799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086533400000000005,"stat_sum":{"num_bytes":49211509,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":20,"num_read":8758,"num_read_kb":23324,"num_write":1134,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.f","version":"136'395","reported_seq":4655,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.263045+0000","last_change":"2024-09-13T09:09:35.263601+0000","last_active":"2024-09-13T09:09:35.263045+0000","last_peered":"2024-09-13T09:09:35.263045+0000","last_clean":"2024-09-13T09:09:35.263045+0000","last_became_active":"2024-09-13T09:09:16.091543+0000","last_became_peered":"2024-09-13T09:09:16.091543+0000","last_unstale":"2024-09-13T09:09:35.263045+0000","last_undegraded":"2024-09-13T09:09:35.263045+0000","last_fullsized":"2024-09-13T09:09:35.263045+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:43:12.463442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024708920000000001,"stat_sum":{"num_bytes":42582629,"num_objects":52,"num_object_clones":9,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":21,"num_read":4521,"num_read_kb":16837,"num_write":371,"num_write_kb":6348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":9699455,"num_keys_recovered":77,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.10","version":"138'447","reported_seq":2478,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.289324+0000","last_change":"2024-09-13T09:09:35.289324+0000","last_active":"2024-09-13T09:09:35.289324+0000","last_peered":"2024-09-13T09:09:35.289324+0000","last_clean":"2024-09-13T09:09:35.289324+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:09:35.289324+0000","last_undegraded":"2024-09-13T09:09:35.289324+0000","last_fullsized":"2024-09-13T09:09:35.289324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058532400000000002,"stat_sum":{"num_bytes":58050848,"num_objects":55,"num_object_clones":6,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":21,"num_read":2598,"num_read_kb":16915,"num_write":439,"num_write_kb":9783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.11","version":"137'1041","reported_seq":3848,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.261813+0000","last_change":"2024-09-13T09:09:35.261813+0000","last_active":"2024-09-13T09:09:35.261813+0000","last_peered":"2024-09-13T09:09:35.261813+0000","last_clean":"2024-09-13T09:09:35.261813+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:09:35.261813+0000","last_undegraded":"2024-09-13T09:09:35.261813+0000","last_fullsized":"2024-09-13T09:09:35.261813+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001458198,"stat_sum":{"num_bytes":39740366,"num_objects":47,"num_object_clones":7,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":15,"num_read":7271,"num_read_kb":11910,"num_write":1535,"num_write_kb":5351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"7"}]},{"pgid":"3.12","version":"136'637","reported_seq":4183,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-09-13T09:09:34.250042+0000","last_change":"2024-09-13T09:09:34.250042+0000","last_active":"2024-09-13T09:09:34.250042+0000","last_peered":"2024-09-13T09:09:34.250042+0000","last_clean":"2024-09-13T09:09:34.250042+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:09:34.250042+0000","last_undegraded":"2024-09-13T09:09:34.250042+0000","last_fullsized":"2024-09-13T09:09:34.250042+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001108511,"stat_sum":{"num_bytes":40999502,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":15,"num_read":5526,"num_read_kb":13814,"num_write":915,"num_write_kb":11369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2e","length":"6"}]},{"pgid":"3.13","version":"138'510","reported_seq":3481,"reported_epoch":138,"state":"active+clean","last_fresh":"2024-09-13T09:09:35.393114+0000","last_change":"2024-09-13T09:09:35.262238+0000","last_active":"2024-09-13T09:09:35.393114+0000","last_peered":"2024-09-13T09:09:35.393114+0000","last_clean":"2024-09-13T09:09:35.393114+0000","last_became_active":"2024-09-13T09:09:04.005219+0000","last_became_peered":"2024-09-13T09:09:04.005219+0000","last_unstale":"2024-09-13T09:09:35.393114+0000","last_undegraded":"2024-09-13T09:09:35.393114+0000","last_fullsized":"2024-09-13T09:09:35.393114+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:56.239198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001688087,"stat_sum":{"num_bytes":21216788,"num_objects":59,"num_object_clones":14,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":27,"num_read":3162,"num_read_kb":13323,"num_write":560,"num_write_kb":7562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1510696075,"num_objects":1560,"num_object_clones":331,"num_object_copies":3120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1560,"num_whiteouts":617,"num_read":199615,"num_read_kb":680904,"num_write":22970,"num_write_kb":337553,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1769,"num_bytes_recovered":174938849,"num_keys_recovered":405,"num_objects_omap":46,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381493248,"data_stored":417786673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18521,"ondisk_log_size":18521,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":64424509475,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":145596,"kb_used_data":70184,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94226244,"statfs":{"total":96636764160,"available":96487673856,"internally_reserved":0,"allocated":71868416,"data_stored":81084474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":77199816},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.56799999999999995,"15min":0.55700000000000005},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.115,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.63100000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.27200000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.121,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66000000000000003,"15min":0.65700000000000003},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63,"15min":0.63},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.071,"5min":1.444,"15min":1.444},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.59399999999999997,"15min":0.57199999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.76400000000000001,"15min":0.76000000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85199999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2989999999999999,"5min":1.343,"15min":1.343},"last":0.92500000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.873,"15min":0.86499999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.92200000000000004,"15min":0.90600000000000003},"min":{"1min":0.58299999999999996,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.92500000000000004,"15min":0.92100000000000004},"min":{"1min":0.52600000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.98599999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509474,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":126908,"kb_used_data":51456,"kb_used_omap":1,"kb_used_meta":75390,"kb_avail":94244932,"statfs":{"total":96636764160,"available":96506810368,"internally_reserved":0,"allocated":52690944,"data_stored":54128918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77199814},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.226},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73999999999999999,"15min":0.751},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.81799999999999995}]},{"osd":1,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80600000000000005,"15min":0.79900000000000004},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.35199999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.077},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.88100000000000001,"15min":0.91400000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.133,"5min":1.401,"15min":1.401},"last":1.034}]},{"osd":4,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":1.0109999999999999,"15min":1.0389999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.998,"5min":1.998,"15min":1.998},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.93600000000000005,"15min":0.95499999999999996},"min":{"1min":0.45700000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.492,"5min":1.651,"15min":1.651},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.98299999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.3180000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.052,"15min":1.0800000000000001},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3049999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.177},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.107,"15min":1.131},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5840000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.3759999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":1.256}]}]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83972,"kb_used_data":56952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94287868,"statfs":{"total":96636764160,"available":96550776832,"internally_reserved":0,"allocated":58318848,"data_stored":62687604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.221,"15min":0.221},"max":{"1min":2.2650000000000001,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.442},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.625,"15min":0.64400000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85899999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.32700000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.27700000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.995,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.68999999999999995,"15min":0.70699999999999996},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.036,"5min":1.214,"15min":1.214},"last":0.39300000000000002}]},{"osd":3,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.30499999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96899999999999997,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.748,"15min":0.76500000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.123,"5min":1.407,"15min":1.407},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.84399999999999997,"15min":0.84799999999999998},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.121,"5min":1.415,"15min":1.415},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.92200000000000004,"15min":0.94199999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2270000000000001,"5min":1.448,"15min":1.448},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96599999999999997,"15min":0.98599999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.224,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.94699999999999995,"15min":0.96299999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2010000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.91100000000000003}]},{"osd":6,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.47399999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.353,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.878,"5min":1.0309999999999999,"15min":1.0569999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.3460000000000001,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":0.82899999999999996}]}]},{"osd":0,"up_from":0,"seq":64424509476,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65788,"kb_used_data":34672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94306052,"statfs":{"total":96636764160,"available":96569397248,"internally_reserved":0,"allocated":35504128,"data_stored":39526677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77700000000000002,"15min":0.76400000000000001},"min":{"1min":0.47499999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.82199999999999995,"15min":0.79700000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":5.4219999999999997,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.622},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.92900000000000005,"15min":0.92800000000000005},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.706,"5min":2.706,"15min":2.706},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":0.94899999999999995,"15min":0.90100000000000002},"min":{"1min":0.53700000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":8.3499999999999996,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.206,"5min":0.98199999999999998,"15min":0.94399999999999995},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":8.2989999999999995,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":1.0169999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.95099999999999996,"15min":0.95299999999999996},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.472,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.92500000000000004,"15min":0.92300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.93500000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.99199999999999999,"15min":0.998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.218,"5min":1.7,"15min":1.7},"last":1.1439999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":1.085,"15min":1.093},"min":{"1min":0.51600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.181},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0700000000000001,"15min":1.0720000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.202}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.232}]}]},{"osd":4,"up_from":14,"seq":60129542178,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102644,"kb_used_data":67432,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94269196,"statfs":{"total":96636764160,"available":96531656704,"internally_reserved":0,"allocated":69050368,"data_stored":74907993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83599999999999997,"15min":0.84599999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.87}]},{"osd":1,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.115,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.212,"5min":1.3,"15min":1.3},"last":1.0640000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.84799999999999998,"15min":0.81999999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":3.5819999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.90900000000000003,"15min":0.91600000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.329}]},{"osd":3,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.873,"15min":0.85299999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.6749999999999998,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.288},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.96699999999999997,"15min":0.97399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.4550000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.597},{"interface":"front","average":{"1min":1.0209999999999999,"5min":0.88900000000000001,"15min":0.86699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.7639999999999998,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.73}]},{"osd":6,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.516},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.008,"15min":1.028},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.867}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.649},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.8080000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":119556,"kb_used_data":62168,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94252284,"statfs":{"total":96636764160,"available":96514338816,"internally_reserved":0,"allocated":63660032,"data_stored":65829103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":1.1200000000000001,"15min":1.179},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0489999999999999,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.753}]},{"osd":1,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.085,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.218,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.0189999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":1.1080000000000001,"15min":1.159},"min":{"1min":0.54900000000000004,"5min":0.435,"15min":0.435},"max":{"1min":1.151,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.878,"15min":0.89100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.84299999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.173,"15min":1.2270000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4910000000000001,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.96999999999999997,"15min":0.97799999999999998},"min":{"1min":0.61299999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3660000000000001,"5min":1.585,"15min":1.585},"last":0.98999999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.0169999999999999,"15min":1.0309999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.383,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.357,"15min":1.415},"min":{"1min":0.56899999999999995,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3799999999999999,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":0.88300000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.99299999999999999,"15min":0.98899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.0800000000000001,"15min":1.091},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.4710000000000001,"5min":1.899,"15min":1.899},"last":1.0900000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.0660000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6295552,"data_stored":7615357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56078336,"data_stored":61319988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51044352,"data_stored":53351670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":70217728,"data_stored":80307207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67407872,"data_stored":74136862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61456384,"data_stored":63431853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":33865728,"data_stored":38755724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":35127296,"data_stored":38868012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T09:09:37.945 INFO:tasks.thrashosds.thrasher:Setting 3.b to [2, 1] 2024-09-13T09:09:37.945 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.b', '2', '1'] 2024-09-13T09:09:37.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.b 2 1 2024-09-13T09:09:37.946 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:37.946 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:37.964 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:37.965 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:37.981 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:37.976+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:38.060 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:38.060 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:38.060 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:38.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:38.082 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:38.076+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:38.183 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:38.177+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:38.192 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 154.05903119499999, 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:09:38.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:09:38.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:09:38.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 105.230749015, 2024-09-13T09:09:38.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:09:38.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.938309176999994, 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:09:38.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:09:38.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.938229379000006, 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:38.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:09:38.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.937980307999993, 2024-09-13T09:09:38.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:38.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.937667167000001, 2024-09-13T09:09:38.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.224 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.937213935000003, 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:09:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.937189420999999, 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:09:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:09:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.936924192000006, 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:09:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.936636343000004, 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:09:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.936175039000005, 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:09:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.997641690000002, 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:09:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:09:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.986648369999997, 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:09:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:09:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.982666471999998, 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:09:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.980038477999997, 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:09:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:09:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:09:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.978485694, 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:09:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:09:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.977232377999997, 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:09:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:09:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.976130345000001, 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:09:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:09:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:09:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.974919767999999, 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:09:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:09:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3:f1cafc87:::rbd_data.1124651f5f97.000000000000004a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 557056~22319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.067224353, 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97276468599999999, 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:09:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092738+0000", 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97003675 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092877+0000", 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139245 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093029+0000", 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015224999999999999 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093847+0000", 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081788800000000001 2024-09-13T09:09:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095250+0000", 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014028529999999999 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095282+0000", 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1961000000000001e-05 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095303+0000", 2024-09-13T09:09:38.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:09:38.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.311 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:38.311 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:38.311 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:38.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:38.314 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.b pg_upmap mapping to [2,1] 2024-09-13T09:09:38.314 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:38.308+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:38.413 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:38.409+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:38.455 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:38.456 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:38.456 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:38.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:38.517 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:38.512+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:38.598 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:38.598 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:38.598 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:38.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:38.657 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:38.652+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:38.715 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:38.708+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:38.788 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:38.788 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.760203098000005, 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:09:38.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:09:38.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.760155764999993, 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:09:38.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:09:38.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.798 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.760098228999993, 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:09:38.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:09:38.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.760046743000004, 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:38.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:09:38.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.759999801000006, 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:09:38.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:09:38.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:09:38.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:38.812 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.759952408000004, 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:09:38.816 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.759905133000004, 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.817 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:38.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:09:38.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.821 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.759858933000004, 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:09:38.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:09:38.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.759794928000005, 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:09:38.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:09:38.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.757323838999994, 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.831 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:09:38.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.835 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.757268116999995, 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:09:38.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:09:38.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.757218214000005, 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:09:38.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.844 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.757171158000006, 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:09:38.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.757122675000005, 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:38.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:09:38.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:38.854 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.757057410000002, 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:09:38.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:09:38.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.757010493999999, 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.859 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:09:38.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.863 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.756923045999997, 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.864 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.755461197000002, 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.869 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:09:38.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.873 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.755383111, 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:09:38.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:38.878 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.755331022000007, 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:38.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:09:38.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:38.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:09:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:09:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:38.883 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:38.883 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:38.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:38.885 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:38.880+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:38.986 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:38.981+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:39.061 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:39.062 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:39.062 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:39.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:39.086 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:39.081+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:39.189 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:39.185+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:39.196 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:39.196 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:39.196 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:39.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:39.290 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:39.285+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:39.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:09:39.338 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:39.338 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:39.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:39.338 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:39.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.338 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.717995309, 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:09:39.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:09:39.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.717971979, 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.343 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.347 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.71794896199999, 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:39.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:09:39.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:09:39.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.351 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.71792583200001, 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.352 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.715309069, 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:09:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:39.360 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.715284983, 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.364 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.71525833, 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.365 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:09:39.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.369 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.71523469, 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.71141914899999, 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:09:39.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:09:39.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:09:39.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.378 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.711395606, 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:09:39.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:09:39.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.383 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.71137231599999, 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:09:39.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.71134880300001, 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.388 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:39.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:39.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:09:39.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.395 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.711290347, 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:09:39.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:09:39.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.400 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.707259411, 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:39.405 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 106.387361114, 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:09:39.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:09:39.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 106.384989335, 2024-09-13T09:09:39.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:09:39.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:09:39.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:39.415 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 106.38445541999999, 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:39.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:09:39.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 106.381054897, 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:09:39.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:09:39.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:39.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 106.38005509, 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:09:39.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 106.376884812, 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.430 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:39.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:09:39.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.435 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:39.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:39.438 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:39.433+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:39.538 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:39.533+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:39.565 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:39.565 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:39.565 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:39.565 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:39.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:39.639 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:39.637+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:39.697 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:39.697 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:39.698 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:39.698 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:39.698 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:39.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:39.740 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:39.737+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:39.833 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.39038149, 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.390356907, 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:09:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.390332903, 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:09:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:09:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.39030941999999, 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:09:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:09:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.39028601299999, 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:09:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.390262764, 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.39023065799999, 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.390195041, 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:09:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.390163061, 2024-09-13T09:09:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:09:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.39009378199999, 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:09:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.390068722, 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:09:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.390045292, 2024-09-13T09:09:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:09:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.390021909, 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:09:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.38999849299999, 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:09:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.38997457000001, 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.389941486, 2024-09-13T09:09:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:09:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:09:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.389917686, 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:09:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.38989336, 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:09:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.38986079999999, 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 124.389820927, 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:39.929 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:39.931 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:09:39.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:39.932 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:39.925+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:39.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:40.032 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:40.024+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:40.102 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:40.103 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:40.103 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:40.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:40.133 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:40.129+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:40.139 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:40.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:40.156 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:40.172 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:40.172 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:40.188 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:40.207 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:40.207 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:40.226 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:40.226 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:40.234 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:40.228+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:40.246 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:40.246 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:40.271 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:40.271 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:40.280 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:40.280 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:40.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:40.341 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:40.336+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:40.436 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:40.428+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:40.477 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:40.478 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:40.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:40.543 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:40.541+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:40.638 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:40.632+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:40.642 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:40.643 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:40.659 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:40.740 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:40.732+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:40.823 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:40.823 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:40.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:40.841 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:40.836+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:40.941 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:40.937+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:40.957 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:40.957 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:40.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:41.042 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:41.036+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:41.134 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:41.134 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:41.144 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:41.141+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:41.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:41.245 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:41.240+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:41.318 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:41.318 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:41.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:41.383 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:41.381+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:41.445 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:41.448 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:41.449 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:41.445+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:41.458 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:41.551 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:41.545+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:41.622 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:41.622 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:41.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:41.680 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:41.677+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:41.751 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:41.749+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:41.804 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:41.804 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:41.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:41.852 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:41.848+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:41.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:09:41.953 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:41.952+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:41.987 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:41.987 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:42.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:42.054 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:42.048+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:42.121 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:42.121 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:42.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:42.179 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:42.177+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:42.255 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:42.255 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:42.255 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:42.253+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:42.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:42.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:42.302 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:09:42.356 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:42.352+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:42.435 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:42.435 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:42.451 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:42.452 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:42.452 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:42.457 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:42.453+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:42.469 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:42.469 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:42.484 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:42.484 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:42.500 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:42.500 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:42.517 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:42.518 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:42.535 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:42.535 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:42.558 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:42.553+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:42.587 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:42.588 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:42.588 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:42.588 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:42.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:42.659 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:42.652+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:42.729 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:42.729 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:42.729 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:42.729 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:42.729 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:42.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:42.759 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:42.757+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:42.760 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:42.760 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:42.781 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:42.781 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:42.861 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:42.856+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93535 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5660672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887890967000004, 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099170151999999998, 2024-09-13T09:09:42.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93535 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024015+0000", 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024014+0000", 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024020+0000", 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3500000000000002e-06 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024038+0000", 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.717e-05 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024051+0000", 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3560000000000001e-05 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.122910+0000", 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098859274999999996 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123058+0000", 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014725 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123185+0000", 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012745000000000001 2024-09-13T09:09:42.918 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93536 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5685248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887843634000006, 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099391837999999996, 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.919 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93536 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024062+0000", 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024061+0000", 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024067+0000", 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6999999999999996e-06 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024084+0000", 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7010000000000001e-05 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024097+0000", 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3233e-05 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123244+0000", 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099147009999999994 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123336+0000", 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1349999999999998e-05 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123454+0000", 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118422 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.923 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887786098000007, 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:09:42.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:42.928 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887734612000003, 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.932 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887687670000005, 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.933 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887640277000003, 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:09:42.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887593002000003, 2024-09-13T09:09:42.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:09:42.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.946 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887546802000003, 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:09:42.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:09:42.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.887482797000004, 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:09:42.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:09:42.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.885011707999993, 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.884955985999994, 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.961 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:09:42.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.884906083000004, 2024-09-13T09:09:42.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:09:42.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.884859027000005, 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:09:42.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:09:42.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.884810544000004, 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:09:42.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.884745279000001, 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.980 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:09:42.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.884698362999998, 2024-09-13T09:09:42.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:09:42.988 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.884610914999996, 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.989 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:09:42.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:09:42.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.883149066000001, 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:09:42.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:09:42.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.883070979999999, 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:42.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:43.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:09:43.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 94.883018891000006, 2024-09-13T09:09:43.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:43.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.008 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:43.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:43.010 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:43.004+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:43.111 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:43.105+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:43.185 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:43.185 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:43.185 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:43.185 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:43.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:43.212 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:43.209+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:43.313 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:43.309+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:43.314 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1] out_osds: [7, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:43.314 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:09:43.315 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-13T09:09:43.315 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T09:09:43.315 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T09:09:43.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-13T09:09:43.322 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:43.322 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:43.322 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:43.322 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:43.322 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:43.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:43.414 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:43.409+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:43.443 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:43.443 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T09:09:43.443 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:43.512 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:43.512 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:43.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:43.512 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.89199392, 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:09:43.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:09:43.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.517 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.89197059, 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:09:43.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:43.522 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.89194757300001, 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:09:43.526 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.89192444299999, 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.527 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:09:43.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:09:43.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.88930768, 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:09:43.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.536 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.88928359400001, 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.537 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.539 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:09:43.540 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:43.541 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.88925694100001, 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:43.542 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.543 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:09:43.544 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.545 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.88923330099999, 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.546 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.547 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.548 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:09:43.549 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.88541776, 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.551 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.552 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.553 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:09:43.554 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.555 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.885394217, 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:09:43.556 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.557 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:09:43.558 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.559 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:43.560 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.885370927, 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.561 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:09:43.562 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.563 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.564 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.88534741399999, 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.565 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.566 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.567 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.88528895799999, 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.570 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:09:43.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:43.574 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.881258022, 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.575 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 110.561359725, 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:09:43.579 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:09:43.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:43.584 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 110.558987946, 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.586 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:43.589 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 110.558454031, 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.592 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 110.555053508, 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:09:43.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 110.554053701, 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:09:43.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:09:43.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.604 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 110.550883423, 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:09:43.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:09:43.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:43.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.609 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:43.610 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:43.610 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:43.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:43.612 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:43.604+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:43.712 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:43.704+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:43.785 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:43.785 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:43.785 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:43.785 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:43.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:43.812 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:43.809+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:43.913 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:43.908+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:43.973 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:43.973 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:43.973 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:43.973 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:43.973 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:43.989 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:44.014 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:44.009+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:44.117 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:44.112+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71168812299999, 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:09:44.156 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:09:44.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:09:44.160 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71166353999999, 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.161 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:09:44.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.165 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71163953600001, 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:44.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.711616053, 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.170 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:09:44.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.174 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71159264600001, 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:09:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:09:44.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:44.179 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71156939700001, 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:44.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:09:44.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71153729100001, 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.184 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:44.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:44.188 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.711501674, 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:44.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:09:44.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71146969399999, 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.193 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:09:44.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.197 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71140041500001, 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:44.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:09:44.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.711375355, 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.206 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.711351925, 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:09:44.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:09:44.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71132854199999, 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:09:44.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:09:44.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71130512600001, 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:09:44.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.711281203, 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:09:44.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.711248119, 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.225 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:09:44.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.711224319, 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:09:44.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71119999300001, 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:09:44.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71116743300001, 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:09:44.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:09:44.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 128.71112755999999, 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.243 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:09:44.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:09:44.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.247 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.248 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:44.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:44.249 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:44.244+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:44.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:09:44.375 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:44.373+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:44.424 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:44.425 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:44.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:44.485 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:44.480+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:44.551 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:44.549+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:44.567 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:44.567 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:44.567 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:44.567 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:44.567 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:44.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:44.673 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:44.668+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:44.753 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:44.748+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52961777499999, 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:09:44.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.760 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52958692499999, 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:09:44.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:09:44.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.529560758, 2024-09-13T09:09:44.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:09:44.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:09:44.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.529521284, 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:09:44.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:09:44.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:44.774 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52948673, 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:09:44.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52945437400001, 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:09:44.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.783 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.529422103, 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:09:44.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52938806700001, 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:09:44.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52935480299999, 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.793 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:09:44.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.797 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52932256299999, 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:44.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:09:44.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:09:44.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52034247700001, 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.802 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:09:44.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.806 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52030159700001, 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:44.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:09:44.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52026132899999, 2024-09-13T09:09:44.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:09:44.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.520214286, 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.816 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:09:44.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:09:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.520168724, 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.821 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:09:44.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.825 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52007855599999, 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:09:44.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.52003200199999, 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.830 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:09:44.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.51839107800001, 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:09:44.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:09:44.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.518325272, 2024-09-13T09:09:44.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:44.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:09:44.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:09:44.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.835317626999995, 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:44.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:09:44.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:09:44.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:44.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:44.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:09:44.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:09:44.850 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.850 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.850 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.850 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:44.850 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:44.850 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:44.850 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:44.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:44.875 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:09:44.893 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:44.888+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:44.955 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:44.948+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:45.010 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:45.010 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:45.010 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:45.018 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:45.025 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:45.037 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:45.037 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:45.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:45.053 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:45.056 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:45.053+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:45.071 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:45.071 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:45.092 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:45.092 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:45.112 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:45.113 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:45.145 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:45.145 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:45.156 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:45.153+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:45.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:45.258 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:45.256+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:45.329 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:45.329 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:45.337 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:45.337 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:45.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:45.359 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:45.359 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:45.360 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:45.357+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:45.460 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:45.452+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:45.470 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:45.470 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:45.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:45.561 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:45.556+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:45.612 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:45.612 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:45.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:45.671 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:45.669+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:45.746 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:45.747 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:45.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:45.762 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:45.760+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:45.863 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:45.861+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:45.936 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:45.936 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:45.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:45.964 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:45.956+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:46.065 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:46.061+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:46.136 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:46.136 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:46.150 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:46.166 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:46.160+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:46.267 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:46.264+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:46.316 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:46.316 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:46.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:46.368 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:46.364+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:46.455 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:46.455 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:46.469 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:46.465+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:46.470 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:46.569 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:46.565+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:46.640 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:46.640 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:46.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:46.671 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:46.664+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:46.772 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:46.768+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:46.819 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:46.820 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:46.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:46.873 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:46.869+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:46.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:09:46.974 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:46.969+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:46.978 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:46.978 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:46.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:47.075 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:47.068+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:47.125 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:47.125 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:47.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:47.181 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:47.176+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:47.278 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:47.272+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:47.306 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:47.306 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:47.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:47.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:47.409 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:47.404+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:47.480 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:47.472+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:47.490 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:47.490 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:47.504 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:47.530 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:47.531 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:47.547 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:47.547 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:47.563 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:47.563 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:47.580 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:47.581 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:47.581 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:47.577+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:47.600 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:47.600 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:47.616 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:47.616 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:47.634 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:47.634 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:47.652 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:47.652 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:47.676 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:47.676 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:47.676 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:47.681 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:47.676+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:47.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:47.782 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:47.776+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.241751671, 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:09:47.864 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.241728341, 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:09:47.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.24170532400001, 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:09:47.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:09:47.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:09:47.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.24168219399999, 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:09:47.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:47.882 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.239065431, 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.886 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.239041345, 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.887 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:09:47.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.23901469200001, 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.891 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:09:47.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:09:47.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.23899105199999, 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:09:47.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.23517551099999, 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.900 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:09:47.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.235151968, 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.905 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:09:47.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:09:47.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.235128678, 2024-09-13T09:09:47.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:09:47.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:09:47.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.23510516499999, 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:09:47.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:09:47.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.23504670899999, 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:09:47.918 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:09:47.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.922 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.231015773, 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:09:47.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:09:47.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:09:47.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.911117476, 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:09:47.927 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:09:47.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:09:47.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.908745697, 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:09:47.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.908211782, 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.937 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:09:47.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.904811259, 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:47.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.946 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.903811452, 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:09:47.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.951 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.900641174, 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:47.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:47.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:47.956 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:47.958 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:09:47.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:47.959 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:47.952+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:48.058 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:48.053+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:48.140 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:48.140 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:48.141 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:48.141 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:48.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:48.158 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:48.152+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:48.259 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:48.253+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:48.281 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:48.281 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:48.281 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:48.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:48.360 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:48.352+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:48.423 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.98001828599999, 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.424 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:48.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97999370299999, 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:09:48.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:09:48.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97996969900001, 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:09:48.433 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:09:48.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.437 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.979946216, 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.438 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:09:48.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.442 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97992280899999, 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97989956000001, 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:09:48.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:09:48.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.451 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97986745399999, 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97983183700001, 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:09:48.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97979985699999, 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.461 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:09:48.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.465 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97973057799999, 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:09:48.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:09:48.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.470 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.979705518, 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:09:48.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.979682088, 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:09:48.475 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:09:48.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.479 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97965870499999, 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.480 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:09:48.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:48.484 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97963528899999, 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:48.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:09:48.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.979611366, 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.489 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:09:48.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.493 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97957828200001, 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:09:48.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:09:48.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.979554482, 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.498 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:48.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:09:48.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.502 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97953015600001, 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:09:48.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.97949759599999, 2024-09-13T09:09:48.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:09:48.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.510 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.511 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 132.979457723, 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:48.512 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:09:48.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:09:48.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.516 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:48.517 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:48.517 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:48.517 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1] out_osds: [7, 0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:48.517 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:09:48.517 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-13T09:09:48.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-13T09:09:48.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:48.519 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:48.512+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:48.619 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:48.612+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:48.692 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:48.692 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:48.692 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:48.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:48.720 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:48.717+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:48.821 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:48.817+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:48.839 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-13T09:09:48.833+0000 7f426cbbd640 -1 mon.a@0(leader).osd e151 definitely_dead 0 2024-09-13T09:09:48.877 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:48.878 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:48.878 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:48.878 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:48.878 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:48.891 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.7. 2024-09-13T09:09:48.897 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:48.918 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-13T09:09:48.945 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:48.940+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:49.016 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:49.016 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:49.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:49.016 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:49.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.016 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14091 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1900544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.79106225699999, 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060180493000000002, 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14091 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.017 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222626+0000", 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222635+0000", 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1260000000000004e-06 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222641+0000", 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2899999999999999e-06 2024-09-13T09:09:49.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222649+0000", 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9820000000000008e-06 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282613+0000", 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059963681999999997 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282691+0000", 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8263999999999995e-05 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282807+0000", 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115555 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.021 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.79103140699999, 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:09:49.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.027 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.79100524, 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:49.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.790965766, 2024-09-13T09:09:49.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:09:49.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.790931212, 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.037 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:09:49.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.79089885600001, 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:09:49.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:09:49.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.790866585, 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:09:49.046 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:09:49.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:09:49.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.79083254899999, 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.051 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:09:49.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.055 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.79079928499999, 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:09:49.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.790767045, 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.060 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:49.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.78178695899999, 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:09:49.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.78174607899999, 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:09:49.069 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:09:49.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.073 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.78170581099999, 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:09:49.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.078 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.781658768, 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:09:49.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.781613206, 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.083 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:49.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.78152303799999, 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:09:49.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.781476484, 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.097 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.77983556000001, 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:09:49.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:49.102 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.779769754, 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:49.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.096762108999997, 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:09:49.107 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:49.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:09:49.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:09:49.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:09:49.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:09:49.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:49.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:49.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:49.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:09:49.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:09:49.113 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.113 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.114 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.114 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:49.114 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:49.114 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:49.115 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:49.108+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:49.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:49.216 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:49.208+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:49.250 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:49.250 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:49.250 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:49.264 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:49.316 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:49.312+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:49.382 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:49.382 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:49.382 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:49.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:49.417 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:49.412+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:49.517 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:49.517 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:49.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:49.517 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.402564228, 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.518 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:09:49.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:09:49.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.40254346099999, 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:09:49.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:09:49.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:09:49.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:09:49.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.40252560899999, 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:09:49.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.532 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.40250904299999, 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.533 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:09:49.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.537 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.40248345800001, 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:09:49.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.402464291, 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.542 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:09:49.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.546 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.402447008, 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:09:49.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:09:49.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.401918138, 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:09:49.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:09:49.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.401886233, 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:09:49.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:09:49.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.401861285, 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.40155672, 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.566 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.399794297, 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:09:49.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.399773447, 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:09:49.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.399751123, 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:09:49.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:09:49.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.399732434, 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:09:49.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.399715382, 2024-09-13T09:09:49.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.39969835799999, 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:09:49.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.389685371, 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:09:49.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:09:49.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 68.009530847999997, 2024-09-13T09:09:49.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:09:49.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 68.009350425999997, 2024-09-13T09:09:49.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:09:49.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:49.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:49.613 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:49.614 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:49.512+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:49.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:49.618 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:49.613+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:49.671 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:49.719 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:49.717+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:49.749 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:49.749 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:49.749 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:49.749 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:49.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:49.817 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:49.817 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:49.819 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:49.812+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:49.850 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:49.850 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:49.867 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:49.867 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:49.890 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:49.911 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:49.911 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:49.921 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:49.916+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:49.932 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:49.932 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:49.939 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:49.939 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:49.953 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:49.953 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:49.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:49.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:09:49.974 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:49.975 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:50.035 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:50.033+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:50.094 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:50.094 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:50.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:50.155 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:50.153+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:50.224 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:50.216+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:50.276 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:50.276 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:50.290 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:50.336 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:50.333+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:50.410 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:50.410 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:50.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:50.425 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:50.420+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:50.527 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:50.521+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:50.593 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:50.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:50.652 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:50.649+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:50.729 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:50.725+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:50.778 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:50.778 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:50.793 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:50.830 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:50.824+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:50.879 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:09:50.947 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:50.945+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:50.975 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:50.975 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:50.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:51.032 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:09:51.024+0000 7f4e9cec6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26342) UID: 0 2024-09-13T09:09:51.133 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:51.129+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:51.165 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:51.166 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:51.180 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:51.234 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:51.228+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:51.321 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:51.321 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:51.335 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:51.328+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:51.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:51.435 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:51.433+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:51.532 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:51.532 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:51.536 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:51.533+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:51.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:51.637 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:51.632+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:51.677 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:51.677 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:51.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:51.737 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:51.732+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:51.838 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:51.833+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:51.885 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:51.886 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:51.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:51.952 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:51.948+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:51.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:09:52.039 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:52.032+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:52.045 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:52.045 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:52.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:52.153 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:52.148+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:52.167 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:52.167 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:52.185 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:52.185 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:52.202 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:52.202 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:52.220 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:52.221 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:52.239 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:52.239 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:52.241 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:52.237+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:52.243 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:52.243 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:52.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:52.266 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:52.266 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:52.289 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:52.289 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:52.306 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:09:52.306 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:09:52.345 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:52.340+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:52.423 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:52.423 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:52.424 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:09:52.424 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:52.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:52.443 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:52.441+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:52.544 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:52.536+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:52.608 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:52.609 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:52.609 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:52.609 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:52.609 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:52.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:52.665 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:52.660+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:52.746 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:52.745+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:52.786 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:52.786 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:52.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:52.786 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:52.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.786 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.786 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:52.786 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342844711, 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:52.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:09:52.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342820128, 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:09:52.791 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.795 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34279612399999, 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:09:52.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:09:52.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:52.800 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34277264100001, 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:52.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.804 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342749234, 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:09:52.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:09:52.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.809 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34272598499999, 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.810 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:09:52.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342693879, 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.815 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:09:52.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.819 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34265826199999, 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342626282, 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342557003, 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.829 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:09:52.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34253194300001, 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:09:52.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.838 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34250851300001, 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.839 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.843 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34248513, 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.844 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:09:52.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:09:52.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.848 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342461714, 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34243779100001, 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:09:52.853 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:09:52.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34240470699999, 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:09:52.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.862 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34238090700001, 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.34235658099999, 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:09:52.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:09:52.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.871 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342324021, 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.872 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:09:52.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.876 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 137.342284148, 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:52.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:52.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:52.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:52.881 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:52.883 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:09:52.883 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:52.877+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:52.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:09:52.983 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:52.977+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:53.026 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:53.026 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:53.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.026 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:163916 3.3 3.4e039283 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-13T09:09:53.026 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:09:52.904242+0000", 2024-09-13T09:09:53.026 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.121126468, 2024-09-13T09:09:53.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.121170471, 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 163916 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.904242+0000", 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.904242+0000", 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.904241+0000", 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.904247+0000", 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.904258+0000", 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1572e-05 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.904267+0000", 2024-09-13T09:09:53.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5350000000000004e-06 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.904361+0000", 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4362000000000002e-05 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.904376+0000", 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4392e-05 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:163918 3.3 3.3faa5c23 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:09:52.908876+0000", 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.11649205999999999, 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.116658435, 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for peered", 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 163918 2024-09-13T09:09:53.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.908876+0000", 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.908876+0000", 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.908875+0000", 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.908883+0000", 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4810000000000006e-06 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.908901+0000", 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8343999999999999e-05 2024-09-13T09:09:53.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.908911+0000", 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6409999999999997e-06 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.908989+0000", 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7764999999999998e-05 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:53.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.909002+0000", 2024-09-13T09:09:53.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2886e-05 2024-09-13T09:09:53.035 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.035 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.035 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.035 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.035 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-13T09:09:53.035 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 2 2024-09-13T09:09:53.035 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:53.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:09:53.085 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:53.080+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:53.168 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:53.168 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:09:53.168 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:09:53.169 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:09:53.169 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:53.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:09:53.184 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:53.181+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:53.285 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:53.281+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:53.356 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:09:53.356 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:09:53.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:09:53.356 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:09:53.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.356 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14092 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2342912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.356 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.12933376699999, 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443906999999998, 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14092 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222657+0000", 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222656+0000", 2024-09-13T09:09:53.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222663+0000", 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0560000000000001e-06 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222671+0000", 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3769999999999996e-06 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222676+0000", 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282840+0000", 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164401999999999 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.282962+0000", 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012136199999999999 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283101+0000", 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013914999999999999 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14093 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2490368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.361 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.1293076, 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060741503000000002, 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14093 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:53.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222683+0000", 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222686+0000", 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.332e-06 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222697+0000", 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1228e-05 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222707+0000", 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9240000000000002e-06 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283151+0000", 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060443494 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283224+0000", 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2989000000000002e-05 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283425+0000", 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020100000000000001 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14094 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.129268126, 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060968300000000003, 2024-09-13T09:09:53.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14094 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222723+0000", 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222714+0000", 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999919 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222727+0000", 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2415999999999999e-05 2024-09-13T09:09:53.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222740+0000", 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2842e-05 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222749+0000", 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0019999999999995e-06 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283466+0000", 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060717750000000001 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283531+0000", 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4091999999999996e-05 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283691+0000", 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000160466 2024-09-13T09:09:53.370 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.371 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.371 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14095 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3080192~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.371 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:53.371 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.129233572, 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061207873000000003, 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14095 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222757+0000", 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222761+0000", 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1640000000000003e-06 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222773+0000", 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-05 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222782+0000", 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9959999999999999e-06 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283750+0000", 2024-09-13T09:09:53.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060967735000000002 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283822+0000", 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2570000000000005e-05 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.283965+0000", 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142874 2024-09-13T09:09:53.376 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.377 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.377 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.377 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14096 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3522560~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.377 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.12920121600001, 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061414448000000003, 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14096 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.378 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222790+0000", 2024-09-13T09:09:53.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222789+0000", 2024-09-13T09:09:53.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:53.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222793+0000", 2024-09-13T09:09:53.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.738e-06 2024-09-13T09:09:53.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222805+0000", 2024-09-13T09:09:53.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1973e-05 2024-09-13T09:09:53.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222814+0000", 2024-09-13T09:09:53.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8880000000000003e-06 2024-09-13T09:09:53.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284000+0000", 2024-09-13T09:09:53.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061186533000000001 2024-09-13T09:09:53.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284059+0000", 2024-09-13T09:09:53.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8678999999999998e-05 2024-09-13T09:09:53.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284204+0000", 2024-09-13T09:09:53.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014524500000000001 2024-09-13T09:09:53.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.387 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.387 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.387 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14097 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3670016~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.387 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:53.388 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.129168945, 2024-09-13T09:09:53.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061639893000000001, 2024-09-13T09:09:53.388 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.389 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.389 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.389 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14097 2024-09-13T09:09:53.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.390 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:53.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222822+0000", 2024-09-13T09:09:53.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222821+0000", 2024-09-13T09:09:53.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222827+0000", 2024-09-13T09:09:53.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0140000000000004e-06 2024-09-13T09:09:53.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222839+0000", 2024-09-13T09:09:53.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2092e-05 2024-09-13T09:09:53.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222848+0000", 2024-09-13T09:09:53.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:09:53.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284246+0000", 2024-09-13T09:09:53.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061398011000000002 2024-09-13T09:09:53.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284322+0000", 2024-09-13T09:09:53.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5700999999999999e-05 2024-09-13T09:09:53.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284462+0000", 2024-09-13T09:09:53.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140037 2024-09-13T09:09:53.399 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.399 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.399 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14098 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4112384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.12913490899999, 2024-09-13T09:09:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061849630000000003, 2024-09-13T09:09:53.400 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14098 2024-09-13T09:09:53.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:53.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222856+0000", 2024-09-13T09:09:53.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222855+0000", 2024-09-13T09:09:53.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:53.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222859+0000", 2024-09-13T09:09:53.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0040000000000001e-06 2024-09-13T09:09:53.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222872+0000", 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2517999999999999e-05 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222881+0000", 2024-09-13T09:09:53.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2860000000000005e-06 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284497+0000", 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061615674000000002 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284556+0000", 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9283999999999997e-05 2024-09-13T09:09:53.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284706+0000", 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014940999999999999 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14127 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1327104~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.408 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.12910164499999, 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061990060999999999, 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14127 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222889+0000", 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222892+0000", 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7560000000000001e-06 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222904+0000", 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1846e-05 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222913+0000", 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6300000000000004e-06 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284742+0000", 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061829149999999999 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284803+0000", 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0656999999999999e-05 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284879+0000", 2024-09-13T09:09:53.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6668000000000005e-05 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14128 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1916928~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.129069405, 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062124335000000003, 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.414 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14128 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222921+0000", 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222920+0000", 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222925+0000", 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7369999999999997e-06 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222942+0000", 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7064000000000001e-05 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.222951+0000", 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8759999999999994e-06 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284913+0000", 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061961563999999997 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.284971+0000", 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8273000000000003e-05 2024-09-13T09:09:53.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.285046+0000", 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4943000000000004e-05 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14782 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 2998272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.419 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.12008931899999, 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060230022000000001, 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14782 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231902+0000", 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231900+0000", 2024-09-13T09:09:53.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231907+0000", 2024-09-13T09:09:53.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.268e-06 2024-09-13T09:09:53.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231922+0000", 2024-09-13T09:09:53.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4866e-05 2024-09-13T09:09:53.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231932+0000", 2024-09-13T09:09:53.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0733999999999999e-05 2024-09-13T09:09:53.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291925+0000", 2024-09-13T09:09:53.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059993112000000001 2024-09-13T09:09:53.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.291987+0000", 2024-09-13T09:09:53.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1883000000000004e-05 2024-09-13T09:09:53.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292132+0000", 2024-09-13T09:09:53.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014430399999999999 2024-09-13T09:09:53.427 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.427 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.427 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.427 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14783 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.428 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:53.428 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.12004843899999, 2024-09-13T09:09:53.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060424833999999997, 2024-09-13T09:09:53.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.428 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.429 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.429 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.429 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.429 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14783 2024-09-13T09:09:53.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.430 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:53.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:53.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231942+0000", 2024-09-13T09:09:53.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231947+0000", 2024-09-13T09:09:53.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5480000000000001e-06 2024-09-13T09:09:53.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231961+0000", 2024-09-13T09:09:53.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4063000000000001e-05 2024-09-13T09:09:53.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231972+0000", 2024-09-13T09:09:53.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0729e-05 2024-09-13T09:09:53.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292169+0000", 2024-09-13T09:09:53.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060196782999999997 2024-09-13T09:09:53.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292228+0000", 2024-09-13T09:09:53.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9001e-05 2024-09-13T09:09:53.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292367+0000", 2024-09-13T09:09:53.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013958699999999999 2024-09-13T09:09:53.438 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.439 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.439 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14784 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3588096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.440 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:53.440 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.12000817099999, 2024-09-13T09:09:53.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060634098999999997, 2024-09-13T09:09:53.440 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.440 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.441 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.441 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.441 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.441 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14784 2024-09-13T09:09:53.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.442 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:53.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231983+0000", 2024-09-13T09:09:53.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231982+0000", 2024-09-13T09:09:53.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.231990+0000", 2024-09-13T09:09:53.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9740000000000002e-06 2024-09-13T09:09:53.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232006+0000", 2024-09-13T09:09:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-13T09:09:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232017+0000", 2024-09-13T09:09:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1072e-05 2024-09-13T09:09:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292409+0000", 2024-09-13T09:09:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060391632000000001 2024-09-13T09:09:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292469+0000", 2024-09-13T09:09:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9638000000000001e-05 2024-09-13T09:09:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292617+0000", 2024-09-13T09:09:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148243 2024-09-13T09:09:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14786 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3735552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:53.451 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.119961128, 2024-09-13T09:09:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060838923000000003, 2024-09-13T09:09:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.452 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.453 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14786 2024-09-13T09:09:53.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.453 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232030+0000", 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232028+0000", 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232036+0000", 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4109999999999999e-06 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232052+0000", 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6402999999999999e-05 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232064+0000", 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2072000000000001e-05 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292658+0000", 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060593979999999999 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292709+0000", 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0869999999999999e-05 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292869+0000", 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159636 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14788 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.458 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.119915566, 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061035161999999997, 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14788 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:53.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232075+0000", 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232074+0000", 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232119+0000", 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4530999999999998e-05 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232139+0000", 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9639999999999998e-05 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232152+0000", 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292904+0000", 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060751597999999997 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.292966+0000", 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1870000000000002e-05 2024-09-13T09:09:53.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293110+0000", 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014466899999999999 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.464 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14817 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1392640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.11982539799999, 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061102765000000003, 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.465 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14817 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232165+0000", 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232164+0000", 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232170+0000", 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-13T09:09:53.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232186+0000", 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6025e-05 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232198+0000", 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:09:53.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293147+0000", 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060949217999999999 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293200+0000", 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3195999999999997e-05 2024-09-13T09:09:53.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293268+0000", 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7930999999999997e-05 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14818 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1982464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.119778844, 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061221538999999998, 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14818 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.471 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232212+0000", 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232211+0000", 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232217+0000", 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6910000000000002e-06 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232233+0000", 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5951e-05 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.232246+0000", 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3652000000000001e-05 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293300+0000", 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061054081000000003 2024-09-13T09:09:53.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293363+0000", 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2657999999999995e-05 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293434+0000", 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0476999999999996e-05 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.475 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14823 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 3751936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.11813792000001, 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060255337999999999, 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.476 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.477 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14823 2024-09-13T09:09:53.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.477 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:53.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233853+0000", 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233852+0000", 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233857+0000", 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3140000000000003e-06 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233873+0000", 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6103000000000002e-05 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233904+0000", 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0108e-05 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293919+0000", 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060015114000000001 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.293974+0000", 2024-09-13T09:09:53.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5173000000000002e-05 2024-09-13T09:09:53.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294108+0000", 2024-09-13T09:09:53.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013439100000000001 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:14839 3.b 3:d744ea22:::rbd_data.1124fee72b32.0000000000000002:9 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 142.118072114, 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060357864999999997, 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:09:53.482 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 14839 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233919+0000", 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233918+0000", 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233944+0000", 2024-09-13T09:09:53.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6310999999999999e-05 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233963+0000", 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9303000000000001e-05 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.233977+0000", 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3857e-05 2024-09-13T09:09:53.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294141+0000", 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.060164084999999999 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294200+0000", 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8430000000000001e-05 2024-09-13T09:09:53.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:31.294277+0000", 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6928000000000006e-05 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.487 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:117459 3.b 3:d41d3c3f:::rbd_data.11241fc9ca17.0000000000000005:head [write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 80.435064468999997, 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.076132277999999998, 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 117459 2024-09-13T09:09:53.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.489 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:53.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916926+0000", 2024-09-13T09:09:53.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916924+0000", 2024-09-13T09:09:53.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:53.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916949+0000", 2024-09-13T09:09:53.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4936999999999999e-05 2024-09-13T09:09:53.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.916984+0000", 2024-09-13T09:09:53.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4992e-05 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.917009+0000", 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5599999999999999e-05 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918223+0000", 2024-09-13T09:09:53.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012131920000000001 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918307+0000", 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4679999999999996e-05 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6", 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.918432+0000", 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124697 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.919155+0000", 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00072305700000000004 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.992955+0000", 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073800350000000001 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993013+0000", 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7810999999999999e-05 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:32.993059+0000", 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5494999999999997e-05 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4213.0:9 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e152)", 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:09:49.139615+0000", 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.2123760649999999, 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.74941406099999996, 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4213", 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2123330464", 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 9 2024-09-13T09:09:53.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.139615+0000", 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.139615+0000", 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:09:53.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.139612+0000", 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999971 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.139632+0000", 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9843e-05 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.139664+0000", 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1922999999999999e-05 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.139679+0000", 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5265999999999999e-05 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.141115+0000", 2024-09-13T09:09:53.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014360919999999999 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.141129+0000", 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4219999999999999e-05 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.888719+0000", 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.74758918500000004 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:09:53.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:49.889029+0000", 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031024200000000002 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:09:53.502 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:09:53.504 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:53.505 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:53.501+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:53.605 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:53.601+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:53.642 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:53.642 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:53.642 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:53.643 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:53.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:53.744 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:53.741+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:53.806 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:53.800+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:53.824 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:53.824 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:53.824 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:53.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:53.876 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:09:53.919 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1, 7] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:53.919 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:09:53.920 INFO:tasks.thrashosds.thrasher:Killing osd 7, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T09:09:53.920 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T09:09:53.920 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:09:53.951 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:53.949+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:53.986 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T09:09:53.986 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.7 2024-09-13T09:09:53.986 INFO:tasks.thrashosds.thrasher:remote for osd 7 is ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:09:53.987 DEBUG:teuthology.orchestra.run.smithi060:> type ceph-objectstore-tool 2024-09-13T09:09:53.992 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T09:09:53.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2024-09-13T09:09:54.008 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:54.005+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:54.022 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:09:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:09:54.022 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.90656719099999, 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:09:54.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:54.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.906546424, 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.906528572, 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:09:54.036 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.906512006, 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:54.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.041 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.906486421, 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:09:54.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:09:54.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.90646725400001, 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:09:54.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:09:54.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.906449971, 2024-09-13T09:09:54.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.905921101, 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:09:54.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:09:54.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.905889196, 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:09:54.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:09:54.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.905864248, 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:09:54.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.90555968300001, 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:09:54.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.90379726, 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:54.078 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.90377641000001, 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:54.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:09:54.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.90375408600001, 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:09:54.083 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:09:54.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.90373539700001, 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:09:54.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:09:54.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.092 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.903718345, 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:09:54.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:09:54.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:54.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.903701321, 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:09:54.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:09:54.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 113.893688334, 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:09:54.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:09:54.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:54.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.513533811000002, 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:09:54.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:09:54.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.513353389000002, 2024-09-13T09:09:54.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:09:54.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:09:54.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:09:54.116 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:54.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:54.118 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:54.113+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:54.219 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:54.212+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:54.291 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:54.291 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:09:54.291 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:54.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:54.319 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:54.312+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:54.329 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:09:54.420 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:54.417+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:54.428 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:09:54.428 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:09:54.428 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:09:54.429 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:09:54.429 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:09:54.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:54.502 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:09:54.502 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:09:54.520 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:09:54.520 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:09:54.521 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:54.516+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:54.536 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:09:54.536 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:09:54.553 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:09:54.553 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:09:54.573 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:09:54.574 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:09:54.599 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:09:54.599 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:09:54.611 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:54.611 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:54.619 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:09:54.619 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:09:54.622 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:54.616+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:54.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:54.722 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:54.720+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:54.793 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:54.793 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:54.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:54.851 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:54.849+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:54.924 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:54.921+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:54.937 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:54.937 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:54.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:55.024 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:55.020+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:55.125 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:55.125 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:55.125 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:55.124+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:55.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:55.227 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:55.224+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:55.264 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:55.264 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:55.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:55.327 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:55.320+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout:3.c 2024-09-13T09:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout:3.8 2024-09-13T09:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout:3.6 2024-09-13T09:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout:3.3 2024-09-13T09:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout:3.0 2024-09-13T09:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout:2.1 2024-09-13T09:09:55.400 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:55.433 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:55.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:55.435 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:55.429+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:55.536 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:55.532+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:55.607 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:55.607 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:55.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:55.637 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:55.633+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:55.738 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:55.732+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:55.745 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:55.745 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:55.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:55.849 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:55.844+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:55.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op export --pgid 3.8 --file /var/log/ceph/exp.3.8.7 2024-09-13T09:09:55.904 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:09:55.929 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:55.929 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:55.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:55.951 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:55.949+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:56.040 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:56.032+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:56.100 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:56.100 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:56.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:56.141 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:56.137+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:56.242 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:56.237+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:56.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:09:56.249+0000 7fec0ffff640 0 --2- 172.21.15.31:0/3531494598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec10070b90 0x7fec1013a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T09:09:56.288 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:56.288 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:56.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:56.353 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:56.348+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:56.424 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:56.425 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:56.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:56.444 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:56.441+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:56.545 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:56.541+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:56.595 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:56.595 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:56.609 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:09:56.645 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:56.641+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:56.726 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:56.727 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:56.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:09:56.746 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:56.740+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:56.847 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:56.841+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:56.873 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:56.873 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:56.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:09:56.947 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:56.940+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:56.955 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:09:57.020 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:57.020 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:57.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:09:57.080 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:57.077+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:57.093 INFO:teuthology.orchestra.run.smithi060.stderr:Exporting 3.8 info 3.8( v 157'844 (0'0,157'844] local-lis/les=156/157 n=37 ec=22/22 lis/c=156/156 les/c/f=157/157/0 sis=156) 2024-09-13T09:09:57.110 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:103f9a31:::rbd_data.1124a9ded6b2.0000000000000044:head# 2024-09-13T09:09:57.110 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:1079096c:::rbd_data.1124f4b52ae4.0000000000000022:head# 2024-09-13T09:09:57.110 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:108cf06b:::rbd_data.1124f4b52ae4.000000000000002e:head# 2024-09-13T09:09:57.110 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:10f2b11e:::rbd_data.1124702db1b3.000000000000001e:head# 2024-09-13T09:09:57.113 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:111865e7:::rbd_data.112498ca9569.000000000000002f:20# 2024-09-13T09:09:57.116 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:111865e7:::rbd_data.112498ca9569.000000000000002f:head# 2024-09-13T09:09:57.116 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:111d3fc1:::rbd_data.1124ea59ddac.000000000000005b:a# 2024-09-13T09:09:57.117 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:111d3fc1:::rbd_data.1124ea59ddac.000000000000005b:head# 2024-09-13T09:09:57.117 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:112e406b:::rbd_data.1124ffa4351.00000000000000ab:head# 2024-09-13T09:09:57.117 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:116a22e1:::rbd_data.11242aa72797.0000000000000031:7# 2024-09-13T09:09:57.118 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:116a22e1:::rbd_data.11242aa72797.0000000000000031:head# 2024-09-13T09:09:57.118 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:126389a0:::rbd_data.1124f4b52ae4.0000000000000003:18# 2024-09-13T09:09:57.119 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:126389a0:::rbd_data.1124f4b52ae4.0000000000000003:head# 2024-09-13T09:09:57.120 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:12e3499c:::rbd_data.112498ca9569.000000000000000a:head# 2024-09-13T09:09:57.120 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:1330803f:::rbd_data.112431c08b5f.0000000000000056:head# 2024-09-13T09:09:57.120 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:133be4d7:::rbd_data.1124a9ded6b2.000000000000011b:head# 2024-09-13T09:09:57.121 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:1358fcf6:::rbd_object_map.1124d4e7fcc5:head# 2024-09-13T09:09:57.121 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:13ccb939:::rbd_data.1124a9ded6b2.00000000000000c0:head# 2024-09-13T09:09:57.121 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:13dc3656:::rbd_data.1124f4b52ae4.000000000000001f:head# 2024-09-13T09:09:57.122 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:1499e676:::rbd_id.image_client.0-clone28:head# 2024-09-13T09:09:57.122 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:15c29231:::rbd_data.1124c678096f.0000000000000009:head# 2024-09-13T09:09:57.122 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:15d97d11:::rbd_data.1124a9ded6b2.000000000000010f:head# 2024-09-13T09:09:57.122 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:15e1a13d:::rbd_data.1124a9ded6b2.0000000000000129:head# 2024-09-13T09:09:57.122 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:15f77e81:::rbd_id.image_client.0-clone18:head# 2024-09-13T09:09:57.122 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:160dfea4:::rbd_data.1124cc1b8ca9.0000000000000003:head# 2024-09-13T09:09:57.124 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:16117e86:::rbd_data.1124ea59ddac.000000000000003d:a# 2024-09-13T09:09:57.124 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:16117e86:::rbd_data.1124ea59ddac.000000000000003d:head# 2024-09-13T09:09:57.124 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:165520ad:::rbd_data.1124a9ded6b2.000000000000006a:head# 2024-09-13T09:09:57.124 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:165d178b:::rbd_data.1124cd332510.0000000000000002:d# 2024-09-13T09:09:57.130 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:165d178b:::rbd_data.1124cd332510.0000000000000002:head# 2024-09-13T09:09:57.131 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:16ca5fe4:::rbd_data.11245864225e.0000000000000000:head# 2024-09-13T09:09:57.142 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:16e3b7ab:::rbd_data.1124a9ded6b2.0000000000000072:head# 2024-09-13T09:09:57.143 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:1721a528:::rbd_data.1124ffa4351.0000000000000010:head# 2024-09-13T09:09:57.143 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:1792e69c:::rbd_data.11241b071601.0000000000000002:head# 2024-09-13T09:09:57.148 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:17a17bcd:::rbd_data.1124d0fab35.0000000000000002:head# 2024-09-13T09:09:57.148 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:57.145+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:17d5923a:::rbd_data.1124a9ded6b2.00000000000000b7:head# 2024-09-13T09:09:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:Read #3:17e81ee6:::rbd_id.image_client.0-clone4:head# 2024-09-13T09:09:57.158 INFO:teuthology.orchestra.run.smithi060.stderr:Export successful 2024-09-13T09:09:57.176 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:57.176 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:57.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:09:57.280 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:57.277+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:57.349 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:57.345+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:57.364 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:57.364 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:57.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:09:57.451 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:57.448+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:57.550 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:57.551 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:57.551 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:57.548+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:57.565 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:09:57.652 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:57.649+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:57.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --force --op remove --pgid 3.8 2024-09-13T09:09:57.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:09:57.793 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:57.788+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:57.855 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:57.849+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:57.879 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:57.880 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:57.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:09:57.984 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:09:57.981+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:09:58.057 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:58.056+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:58.078 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:58.078 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:58.094 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:09:58.179 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:58.177+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:58.258 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:58.256+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:58.260 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:58.260 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:58.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:09:58.363 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:09:58.361+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:09:58.439 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:58.439 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:58.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:09:58.459 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:58.456+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:58.560 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:58.560+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:58.626 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:58.627 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:58.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:09:58.662 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:58.660+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:58.762 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:58.762 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:58.763 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:58.760+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:58.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:09:58.864 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:58.860+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:58.912 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:58.912 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:58.926 INFO:teuthology.orchestra.run.smithi060.stdout: marking collection for removal 2024-09-13T09:09:58.926 INFO:teuthology.orchestra.run.smithi060.stdout:setting '_remove' omap key 2024-09-13T09:09:58.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:09:58.933 INFO:teuthology.orchestra.run.smithi060.stdout:finish_remove_pgs 3.8_head removing 3.8 2024-09-13T09:09:58.943 INFO:teuthology.orchestra.run.smithi060.stdout:Remove successful 2024-09-13T09:09:58.971 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:58.969+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:58.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:09:59.060 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:59.060 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:59.066 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:59.060+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:59.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:09:59.167 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:59.160+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:59.215 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:59.215 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:59.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:09:59.267 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:59.265+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:59.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-13T09:09:59.321+0000 7f0d72ffd640 0 --2- 172.21.15.31:0/3784992909 >> v2:172.21.15.31:3300/0 conn(0x7f0d740a6ef0 0x7f0d7413d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T09:09:59.368 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:09:59.364+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:09:59.391 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:59.391 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:59.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:09:59.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.8.7 2024-09-13T09:09:59.478 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:09:59.472+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:09:59.484 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:09:59.490 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:09:59.490 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:09:59.570 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:59.565+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:59.571 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:59.571 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:59.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:09:59.671 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:09:59.669+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:09:59.703 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:59.703 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:59.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:09:59.772 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:09:59.768+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:09:59.846 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:09:59.846 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:09:59.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:09:59.872 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:59.869+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:59.974 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:09:59.969+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:09:59.991 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:10:00.058 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:00.059 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:00.074 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:00.072+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:00.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:10:00.175 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:00.168+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:00.239 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:00.239 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:00.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:10:00.297 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:00.296+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:00.377 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:00.373+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:00.431 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:00.431 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:00.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:10:00.478 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:00.473+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:00.569 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:00.569 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:00.579 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:00.576+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:00.584 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:10:00.681 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:00.680+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:00.690 INFO:teuthology.orchestra.run.smithi060.stdout:get_pg_num_history pg_num_history pg_num_history(e157 pg_nums {1={15=1},2={18=8},3={22=32}} deleted_pools ) 2024-09-13T09:10:00.691 INFO:teuthology.orchestra.run.smithi060.stdout:Importing pgid 3.8 2024-09-13T09:10:00.697 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:103f9a31:::rbd_data.1124a9ded6b2.0000000000000044:head# 2024-09-13T09:10:00.700 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 25=[]:{} 2024-09-13T09:10:00.701 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:1079096c:::rbd_data.1124f4b52ae4.0000000000000022:head# 2024-09-13T09:10:00.701 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 18=[]:{} 2024-09-13T09:10:00.701 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:108cf06b:::rbd_data.1124f4b52ae4.000000000000002e:head# 2024-09-13T09:10:00.701 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 18=[]:{} 2024-09-13T09:10:00.701 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:10f2b11e:::rbd_data.1124702db1b3.000000000000001e:head# 2024-09-13T09:10:00.702 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.704 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:111865e7:::rbd_data.112498ca9569.000000000000002f:20# 2024-09-13T09:10:00.706 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:111865e7:::rbd_data.112498ca9569.000000000000002f:head# 2024-09-13T09:10:00.706 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 20=[]:{20=[20]} 2024-09-13T09:10:00.706 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:00.707 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:00.707 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:111d3fc1:::rbd_data.1124ea59ddac.000000000000005b:a# 2024-09-13T09:10:00.707 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:111d3fc1:::rbd_data.1124ea59ddac.000000000000005b:head# 2024-09-13T09:10:00.707 INFO:teuthology.orchestra.run.smithi060.stdout:snapset a=[]:{a=[a]} 2024-09-13T09:10:00.708 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:112e406b:::rbd_data.1124ffa4351.00000000000000ab:head# 2024-09-13T09:10:00.708 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.709 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:116a22e1:::rbd_data.11242aa72797.0000000000000031:7# 2024-09-13T09:10:00.710 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:116a22e1:::rbd_data.11242aa72797.0000000000000031:head# 2024-09-13T09:10:00.710 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 7=[]:{7=[7]} 2024-09-13T09:10:00.711 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:126389a0:::rbd_data.1124f4b52ae4.0000000000000003:18# 2024-09-13T09:10:00.712 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:126389a0:::rbd_data.1124f4b52ae4.0000000000000003:head# 2024-09-13T09:10:00.712 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 18=[]:{18=[18]} 2024-09-13T09:10:00.713 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:12e3499c:::rbd_data.112498ca9569.000000000000000a:head# 2024-09-13T09:10:00.713 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 20=[]:{} 2024-09-13T09:10:00.713 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:1330803f:::rbd_data.112431c08b5f.0000000000000056:head# 2024-09-13T09:10:00.713 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.714 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:133be4d7:::rbd_data.1124a9ded6b2.000000000000011b:head# 2024-09-13T09:10:00.714 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 25=[]:{} 2024-09-13T09:10:00.714 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:1358fcf6:::rbd_object_map.1124d4e7fcc5:head# 2024-09-13T09:10:00.714 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.715 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:13ccb939:::rbd_data.1124a9ded6b2.00000000000000c0:head# 2024-09-13T09:10:00.715 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 25=[]:{} 2024-09-13T09:10:00.715 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:13dc3656:::rbd_data.1124f4b52ae4.000000000000001f:head# 2024-09-13T09:10:00.715 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 18=[]:{} 2024-09-13T09:10:00.716 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:1499e676:::rbd_id.image_client.0-clone28:head# 2024-09-13T09:10:00.716 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.716 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:15c29231:::rbd_data.1124c678096f.0000000000000009:head# 2024-09-13T09:10:00.716 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.717 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:15d97d11:::rbd_data.1124a9ded6b2.000000000000010f:head# 2024-09-13T09:10:00.717 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 25=[]:{} 2024-09-13T09:10:00.717 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:15e1a13d:::rbd_data.1124a9ded6b2.0000000000000129:head# 2024-09-13T09:10:00.717 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 25=[]:{} 2024-09-13T09:10:00.717 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:15f77e81:::rbd_id.image_client.0-clone18:head# 2024-09-13T09:10:00.717 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.718 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:160dfea4:::rbd_data.1124cc1b8ca9.0000000000000003:head# 2024-09-13T09:10:00.718 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:10:00.721 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:16117e86:::rbd_data.1124ea59ddac.000000000000003d:a# 2024-09-13T09:10:00.721 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:16117e86:::rbd_data.1124ea59ddac.000000000000003d:head# 2024-09-13T09:10:00.721 INFO:teuthology.orchestra.run.smithi060.stdout:snapset a=[]:{a=[a]} 2024-09-13T09:10:00.722 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:165520ad:::rbd_data.1124a9ded6b2.000000000000006a:head# 2024-09-13T09:10:00.722 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 25=[]:{} 2024-09-13T09:10:00.722 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:165d178b:::rbd_data.1124cd332510.0000000000000002:d# 2024-09-13T09:10:00.728 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:165d178b:::rbd_data.1124cd332510.0000000000000002:head# 2024-09-13T09:10:00.728 INFO:teuthology.orchestra.run.smithi060.stdout:snapset d=[]:{d=[d]} 2024-09-13T09:10:00.729 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:16ca5fe4:::rbd_data.11245864225e.0000000000000000:head# 2024-09-13T09:10:00.734 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.753 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:16e3b7ab:::rbd_data.1124a9ded6b2.0000000000000072:head# 2024-09-13T09:10:00.753 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 25=[]:{} 2024-09-13T09:10:00.754 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:1721a528:::rbd_data.1124ffa4351.0000000000000010:head# 2024-09-13T09:10:00.754 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.755 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:1792e69c:::rbd_data.11241b071601.0000000000000002:head# 2024-09-13T09:10:00.757 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.765 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:17a17bcd:::rbd_data.1124d0fab35.0000000000000002:head# 2024-09-13T09:10:00.769 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.781 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:00.777+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:00.787 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:17d5923a:::rbd_data.1124a9ded6b2.00000000000000b7:head# 2024-09-13T09:10:00.787 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 25=[]:{} 2024-09-13T09:10:00.788 INFO:teuthology.orchestra.run.smithi060.stdout:Write #3:17e81ee6:::rbd_id.image_client.0-clone4:head# 2024-09-13T09:10:00.788 INFO:teuthology.orchestra.run.smithi060.stdout:snapset 0=[]:{} 2024-09-13T09:10:00.788 INFO:teuthology.orchestra.run.smithi060.stdout:write_pg epoch 157 info 3.8( v 157'844 (0'0,157'844] local-lis/les=156/157 n=37 ec=22/22 lis/c=156/156 les/c/f=157/157/0 sis=156) 2024-09-13T09:10:00.793 INFO:teuthology.orchestra.run.smithi060.stdout:Import successful 2024-09-13T09:10:00.849 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:00.849 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:00.864 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:10:00.908 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:00.905+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:00.984 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:00.981+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:01.041 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:01.041 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:01.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:10:01.099 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:01.097+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:01.186 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:01.181+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:01.230 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:01.230 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:01.245 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:10:01.288 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:01.285+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:01.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /var/log/ceph/exp.3.8.7 2024-09-13T09:10:01.387 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:01.384+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:01.412 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:01.412 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:01.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:10:01.488 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:01.485+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:01.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:10:01.548 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:01.548 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:01.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:10:01.607 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:01.605+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:01.634 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:01.635 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:01.651 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:01.651 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:01.669 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:01.669 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:01.685 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:01.685 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:01.689 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:01.685+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:01.691 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:01.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:10:01.706 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:01.707 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:01.727 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:01.728 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:01.748 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:01.748 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:01.748 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:10:01.752 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:10:01.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:10:01.791 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:01.788+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:01.822 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:01.822 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:01.822 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:01.822 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:01.822 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:01.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:10:01.892 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:01.889+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:01.963 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.621499755, 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:01.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:01.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:10:01.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:01.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62146245, 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:01.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:01.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:01.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:10:01.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62144273600001, 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.974 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:01.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:10:01.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:01.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62142641099999, 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:01.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:01.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:10:01.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:10:01.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.621410545, 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:01.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:01.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62139475999999, 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.989 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:01.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:10:01.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:10:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:10:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:10:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:10:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:10:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:10:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:10:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:10:01.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:10:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:01.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62137568899999, 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:01.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:10:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:10:02.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62135914199999, 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.001 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.621331543, 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:02.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:02.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:10:02.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:10:02.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:10:02.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:10:02.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62131430100001, 2024-09-13T09:10:02.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:10:02.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.621283858, 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:10:02.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:10:02.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:10:02.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:02.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.621264943, 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:10:02.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:10:02.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.026 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62124715499999, 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62122193100001, 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:10:02.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:10:02.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62120372699999, 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:10:02.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:02.040 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.621183409, 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:02.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:10:02.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62116486100001, 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.621147587, 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.050 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.62113013300001, 2024-09-13T09:10:02.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:10:02.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:10:02.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:10:02.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:162508 3.b 3.9f9e59eb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.552588368999999, 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.47181327899999997, 2024-09-13T09:10:02.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 162508 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407494+0000", 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407503+0000", 2024-09-13T09:10:02.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0699999999999996e-06 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407523+0000", 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9692999999999999e-05 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407535+0000", 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2689e-05 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407598+0000", 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2064999999999996e-05 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407606+0000", 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9560000000000003e-06 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.879280+0000", 2024-09-13T09:10:02.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.47167379599999998 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.879309+0000", 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8436999999999999e-05 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.065 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:02.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:10:02.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:10:02.067 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:02.061+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:02.168 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:02.164+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:02.241 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:02.242 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:02.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:10:02.308 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:02.305+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:02.369 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:02.368+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:02.376 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:02.376 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:02.376 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:02.376 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:02.377 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:02.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:10:02.475 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:02.473+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:02.561 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 178.42670743100001, 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:10:02.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:10:02.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:10:02.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 129.59842525100001, 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:10:02.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:10:02.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.305985413, 2024-09-13T09:10:02.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.305905615, 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:10:02.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.305656544, 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.584 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:10:02.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:10:02.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.30534340299999, 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:10:02.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:10:02.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.304890171, 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:10:02.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.30486565699999, 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:10:02.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.304600428, 2024-09-13T09:10:02.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:10:02.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.304312579, 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:10:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:10:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:10:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 122.303851275, 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.365317926000003, 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:10:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:10:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.354324606000006, 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:10:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.350342707999999, 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:10:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:10:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:10:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.347714714000006, 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:10:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.346161929999994, 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:10:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:10:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:10:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:10:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.344908614000005, 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:10:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:10:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.343806580999996, 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:10:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:10:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:10:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:10:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 85.342596004000001, 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:10:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3:f1cafc87:::rbd_data.1124651f5f97.000000000000004a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 557056~22319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.434900589000002, 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97276468599999999, 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:10:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:10:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:10:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092738+0000", 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97003675 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092877+0000", 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139245 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093029+0000", 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015224999999999999 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093847+0000", 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081788800000000001 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095250+0000", 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014028529999999999 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095282+0000", 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1961000000000001e-05 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095303+0000", 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:02.677 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:02.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:10:02.678 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:02.676+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:02.779 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:02.777+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:02.812 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:02.812 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:02.812 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:02.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:10:02.880 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:02.880+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:02.952 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:02.952 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:02.952 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:02.953 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:02.953 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:02.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:10:03.004 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:10:03.009 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:03.008+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:03.083 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:03.080+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:03.148 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:03.148 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:03.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:03.148 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.120072077, 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.149 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:10:03.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.120020591, 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.156 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.119973649, 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.158 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.119926256, 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.163 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.164 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:10:03.165 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.166 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.167 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.119878981, 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.168 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.119832781, 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.173 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:10:03.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:10:03.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.119768776, 2024-09-13T09:10:03.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:10:03.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:10:03.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.117297687, 2024-09-13T09:10:03.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:10:03.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.11724196500001, 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.187 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:10:03.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:10:03.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.117192062, 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.192 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:10:03.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:10:03.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:10:03.196 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.117145006, 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:10:03.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.117096523, 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.201 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:10:03.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:10:03.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.117031258, 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:10:03.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.210 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.11698434199999, 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:10:03.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:10:03.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:10:03.215 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.11689689400001, 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:10:03.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.115435045, 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:10:03.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:10:03.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.115356959, 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:10:03.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 115.11530487, 2024-09-13T09:10:03.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:10:03.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:163923 3.8 3:148e04b3:::rbd_data.112498fc7c8e.0000000000000023:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:09:52.910734+0000", 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.233458009, 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98265065900000004, 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:03.234 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 163923 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910734+0000", 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910734+0000", 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910733+0000", 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910738+0000", 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4020000000000001e-06 2024-09-13T09:10:03.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910748+0000", 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0011e-05 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910756+0000", 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6040000000000004e-06 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910821+0000", 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4963999999999997e-05 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910834+0000", 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2717999999999999e-05 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.887268+0000", 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97643416900000002 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.887763+0000", 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049472600000000002 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 7", 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.888079+0000", 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000316235 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.890829+0000", 2024-09-13T09:10:03.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002750574 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.893204+0000", 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023749869999999998 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.893285+0000", 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0971999999999997e-05 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.893385+0000", 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9140000000000003e-05 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:163927 3.8 3:10c4bde7:::rbd_data.112498fc7c8e.0000000000000027:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:09:52.911885+0000", 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 10.232307077, 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98293617899999997, 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.241 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 163927 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911885+0000", 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911885+0000", 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911884+0000", 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911892+0000", 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2130000000000008e-06 2024-09-13T09:10:03.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911906+0000", 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4527e-05 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911919+0000", 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2194e-05 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911979+0000", 2024-09-13T09:10:03.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0331000000000002e-05 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911989+0000", 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.696e-06 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.891043+0000", 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97905409099999996 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.891251+0000", 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020876099999999999 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 7", 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.891447+0000", 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000195632 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.893615+0000", 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002168199 2024-09-13T09:10:03.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.894564+0000", 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000949229 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.894709+0000", 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014445899999999999 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.894821+0000", 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112116 2024-09-13T09:10:03.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.248 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.248 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.248 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.248 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.248 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:03.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:10:03.250 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:03.245+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:03.350 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:03.348+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:03.423 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:03.423 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:03.424 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:03.424 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:03.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:10:03.450 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:03.445+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:03.552 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:03.549+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:03.608 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:03.608 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:03.608 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:03.608 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:03.608 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:03.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:10:03.653 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:03.649+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:03.749 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:03.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12903106799999, 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:10:03.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12900773800001, 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.754 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:03.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:10:03.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12898472099999, 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.759 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:10:03.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:10:03.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12896159100001, 2024-09-13T09:10:03.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:10:03.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.767 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12634482799999, 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.768 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:10:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:10:03.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:10:03.775 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12632074199999, 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.776 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:10:03.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.780 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.126294089, 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:10:03.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.126270449, 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.785 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12245490800001, 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:10:03.793 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12243136500001, 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.794 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:10:03.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.798 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12240807500001, 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:10:03.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.12238456200001, 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:10:03.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.122326106, 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.808 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:10:03.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 149.11829517000001, 2024-09-13T09:10:03.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:10:03.816 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 130.798396873, 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.817 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:10:03.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:10:03.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 130.79602509399999, 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.822 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:10:03.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:10:03.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 130.79549117900001, 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:10:03.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 130.792090656, 2024-09-13T09:10:03.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:10:03.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:10:03.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 130.791090849, 2024-09-13T09:10:03.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:10:03.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 130.787920571, 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:03.842 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:10:03.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:03.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:10:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:10:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:03.847 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:03.847 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:03.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:10:03.849 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:03.845+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:03.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:10:03.949 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:03.945+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:03.978 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:03.979 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:03.979 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:03.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:10:04.007 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:04.008 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:04.028 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:04.028 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:04.046 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:04.047 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:04.050 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:04.045+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:04.065 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:04.065 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:04.084 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:04.085 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:04.105 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:04.105 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:04.122 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:04.122 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:04.122 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:04.124 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:04.124 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:04.125 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:10:04.128 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:10:04.129 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:10:04.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:10:04.171 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:04.169+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:04.252 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:04.248+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:04.262 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:04.262 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:04.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:10:04.354 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:04.352+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:04.408 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:04.408 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:04.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:10:04.455 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:04.452+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:04.546 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:04.546 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:04.556 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:04.553+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:04.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:10:04.657 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:04.656+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:04.727 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:04.727 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:04.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:10:04.788 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:04.785+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:04.858 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:04.853+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:04.860 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:04.860 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:04.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:10:04.968 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:04.965+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:05.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:10:05.060 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:05.056+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:05.060 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:05.060 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:05.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:10:05.161 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:05.160+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:05.262 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:05.260+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:05.272 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:05.272 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:05.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:10:05.363 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:05.357+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:05.402 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:05.402 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:05.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:10:05.503 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:05.501+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:05.564 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:05.560+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:05.583 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:05.583 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:05.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:10:05.665 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:05.664+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:05.762 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:05.762 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:05.766 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:05.764+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:05.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:10:05.867 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:05.864+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:05.894 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:05.895 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:05.909 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:10:05.968 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:05.965+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:06.034 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:10:06.037 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:06.037 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:06.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:10:06.069 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:06.068+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:06.129 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:10:06.170 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:06.168+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:06.236 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:06.236 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:06.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:10:06.271 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:06.265+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:06.299 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:06.299 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:06.316 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:06.316 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:06.331 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:06.331 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:06.348 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:06.349 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:06.366 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:06.367 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:06.372 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:06.372+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:06.386 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:06.386 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:06.389 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:06.389 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:06.389 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:06.389 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:06.404 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:10:06.405 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:06.405 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:10:06.409 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:10:06.410 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:10:06.410 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1, 7] out_osds: [0] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-13T09:10:06.410 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:10:06.410 INFO:tasks.thrashosds.thrasher:Reweighting osd 4 to 0.9430650142853432 2024-09-13T09:10:06.410 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 0.9430650142853432 2024-09-13T09:10:06.500 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:06.497+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:06.563 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:06.563 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:06.563 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:06.564 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:06.564 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:06.574 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:06.572+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:06.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:10:06.676 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:06.672+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:06.744 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:06.744 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:06.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:06.744 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:06.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.744 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93537 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5709824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.744 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.715590061, 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099622485999999996, 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93537 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024120+0000", 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024119+0000", 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024125+0000", 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.443e-06 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024144+0000", 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9156000000000001e-05 2024-09-13T09:10:06.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024159+0000", 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123520+0000", 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099361852000000001 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123633+0000", 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112742 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123742+0000", 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010914899999999999 2024-09-13T09:10:06.749 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93538 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5734400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.715538575, 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099829897000000001, 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.750 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93538 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024171+0000", 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024170+0000", 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024177+0000", 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0460000000000002e-06 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024194+0000", 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742999999999999e-05 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024208+0000", 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3506000000000001e-05 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123788+0000", 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099580718999999998 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.123881+0000", 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2973999999999994e-05 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124001+0000", 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119782 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93539 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5758976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.715491633, 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10014899300000001, 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93539 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024218+0000", 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024217+0000", 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024223+0000", 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-06 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024240+0000", 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7292999999999999e-05 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024254+0000", 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.366e-05 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124074+0000", 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099820289000000006 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124222+0000", 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147856 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124367+0000", 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533499999999999 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93540 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5783552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.71544424, 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100511721, 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93540 2024-09-13T09:10:06.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024266+0000", 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024264+0000", 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024271+0000", 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6869999999999997e-06 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024288+0000", 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7013000000000001e-05 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024302+0000", 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3579e-05 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124460+0000", 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100158737 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124579+0000", 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118668 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124777+0000", 2024-09-13T09:10:06.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000198464 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93541 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5808128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.715396965, 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10082640399999999, 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93541 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024313+0000", 2024-09-13T09:10:06.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024312+0000", 2024-09-13T09:10:06.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024318+0000", 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5199999999999997e-06 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024334+0000", 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436e-05 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024348+0000", 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3587000000000001e-05 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.124854+0000", 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100506148 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125009+0000", 2024-09-13T09:10:06.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015496299999999999 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125139+0000", 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130586 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.772 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93542 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5832704~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.715350765, 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10107108200000001, 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93542 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024359+0000", 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024358+0000", 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024364+0000", 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024382+0000", 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:10:06.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024410+0000", 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125211+0000", 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100800467 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125311+0000", 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100438 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125430+0000", 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119101 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93543 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5857280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.71528676, 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10125526999999999, 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93543 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.778 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024423+0000", 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024422+0000", 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024430+0000", 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5630000000000003e-06 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024448+0000", 2024-09-13T09:10:06.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8196e-05 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.024463+0000", 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.523e-05 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125484+0000", 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102135299999999 2024-09-13T09:10:06.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125573+0000", 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8782000000000005e-05 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.125678+0000", 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105202 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93545 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5906432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.712815671, 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099490538000000003, 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93545 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.783 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026894+0000", 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026893+0000", 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026901+0000", 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3969999999999994e-06 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026922+0000", 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0363000000000001e-05 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026937+0000", 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5591999999999999e-05 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126018+0000", 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099080821999999999 2024-09-13T09:10:06.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126191+0000", 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017262500000000001 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126385+0000", 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019414899999999999 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93546 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5931008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.712759949, 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099695775, 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93546 2024-09-13T09:10:06.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026950+0000", 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026949+0000", 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026956+0000", 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026973+0000", 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7833999999999999e-05 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026988+0000", 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4449e-05 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126451+0000", 2024-09-13T09:10:06.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099462904000000005 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126535+0000", 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3957999999999998e-05 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126646+0000", 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011101 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.792 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93547 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5955584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.712710046, 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099982352999999996, 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93547 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027000+0000", 2024-09-13T09:10:06.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.026999+0000", 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027005+0000", 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1360000000000002e-06 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027022+0000", 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7629999999999999e-05 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027036+0000", 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3546999999999999e-05 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126724+0000", 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099688592000000006 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126878+0000", 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015310200000000001 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.126982+0000", 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104676 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93548 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 5980160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.71266299, 2024-09-13T09:10:06.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100195617, 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93548 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:10:06.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027047+0000", 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027046+0000", 2024-09-13T09:10:06.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027053+0000", 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9229999999999996e-06 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027070+0000", 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7476e-05 2024-09-13T09:10:06.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027083+0000", 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3304e-05 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127044+0000", 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099960249000000001 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127130+0000", 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6644000000000006e-05 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127243+0000", 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112191 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.803 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93549 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6004736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.712614507, 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10047241799999999, 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93549 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027095+0000", 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027116+0000", 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1158999999999999e-05 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027135+0000", 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8902999999999998e-05 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027149+0000", 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127292+0000", 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100142839 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127383+0000", 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1230000000000006e-05 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127568+0000", 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185057 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93550 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6029312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.71254924199999, 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100837128, 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93550 2024-09-13T09:10:06.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027161+0000", 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027160+0000", 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027166+0000", 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027183+0000", 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7263000000000001e-05 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027197+0000", 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3315999999999999e-05 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127645+0000", 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100448445 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127814+0000", 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169023 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.127998+0000", 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000183551 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93551 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6053888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.71250232600001, 2024-09-13T09:10:06.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.101059044, 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93551 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027208+0000", 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027207+0000", 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027212+0000", 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.49e-06 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027232+0000", 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0233000000000001e-05 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027246+0000", 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.385e-05 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128054+0000", 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100807547 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128137+0000", 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3483e-05 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128267+0000", 2024-09-13T09:10:06.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012926800000000001 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93552 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6078464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.712414878, 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10141984599999999, 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93552 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027295+0000", 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027294+0000", 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027301+0000", 2024-09-13T09:10:06.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.763e-06 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027324+0000", 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.281e-05 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027340+0000", 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5366000000000002e-05 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128333+0000", 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100993844 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128535+0000", 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020140100000000001 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128715+0000", 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018005500000000001 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93553 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6103040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.710953029, 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100272238, 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93553 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028757+0000", 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.027350+0000", 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9985933 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028769+0000", 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001418819 2024-09-13T09:10:06.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028798+0000", 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8765999999999999e-05 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028818+0000", 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0407e-05 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128771+0000", 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.099952604 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.128856+0000", 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4666999999999994e-05 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129029+0000", 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017353099999999999 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93554 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6127616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.71087494299999, 2024-09-13T09:10:06.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10060285300000001, 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93554 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:10:06.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028835+0000", 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028834+0000", 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028842+0000", 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9889999999999992e-06 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028860+0000", 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8386999999999999e-05 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028874+0000", 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.452e-05 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129104+0000", 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100229527 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129261+0000", 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015689299999999999 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129438+0000", 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176897 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.835 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:93555 3.1b 3:dd96e71d:::rbd_data.1124943a2b90.0000000000000001:16 [read 6152192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.710822854, 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100890514, 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 93555 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028887+0000", 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028886+0000", 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028893+0000", 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028910+0000", 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7518999999999999e-05 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.028927+0000", 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7277000000000001e-05 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129499+0000", 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.100571668 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129611+0000", 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111858 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:08:08.129778+0000", 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654899999999999 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:163923 3.8 3:148e04b3:::rbd_data.112498fc7c8e.0000000000000023:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-13T09:10:06.840 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:09:52.910734+0000", 2024-09-13T09:10:06.841 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 13.828975993, 2024-09-13T09:10:06.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98265065900000004, 2024-09-13T09:10:06.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.844 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:06.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.844 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.844 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 163923 2024-09-13T09:10:06.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910734+0000", 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910734+0000", 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910733+0000", 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910738+0000", 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4020000000000001e-06 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910748+0000", 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0011e-05 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910756+0000", 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6040000000000004e-06 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910821+0000", 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4963999999999997e-05 2024-09-13T09:10:06.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.910834+0000", 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2717999999999999e-05 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.887268+0000", 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97643416900000002 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.887763+0000", 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049472600000000002 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 7", 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.888079+0000", 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000316235 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.890829+0000", 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002750574 2024-09-13T09:10:06.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.893204+0000", 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023749869999999998 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.893285+0000", 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0971999999999997e-05 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.893385+0000", 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9140000000000003e-05 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:163927 3.8 3:10c4bde7:::rbd_data.112498fc7c8e.0000000000000027:head [delete] snapc 22=[] ondisk+write+known_if_redirected+supports_pool_eio e156)", 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:09:52.911885+0000", 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 13.827825061, 2024-09-13T09:10:06.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98293617899999997, 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 163927 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911885+0000", 2024-09-13T09:10:06.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911885+0000", 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911884+0000", 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911892+0000", 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2130000000000008e-06 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911906+0000", 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4527e-05 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911919+0000", 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2194e-05 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911979+0000", 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0331000000000002e-05 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:52.911989+0000", 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.696e-06 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.891043+0000", 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97905409099999996 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.891251+0000", 2024-09-13T09:10:06.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020876099999999999 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 7", 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.891447+0000", 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000195632 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.893615+0000", 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002168199 2024-09-13T09:10:06.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.894564+0000", 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000949229 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.894709+0000", 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014445899999999999 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:06.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:09:53.894821+0000", 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112116 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:06.859 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:06.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:10:06.861 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:06.857+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:06.962 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:06.960+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:07.034 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:07.035 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:07.035 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:07.035 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:07.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:10:07.053 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 0.943065 (f16c) 2024-09-13T09:10:07.062 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:07.057+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:07.163 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:07.157+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:07.238 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:07.238 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:07.238 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:07.252 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:10:07.297 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:07.296+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:07.365 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:07.364+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.76049872600001, 2024-09-13T09:10:07.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:10:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.386 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.760475396, 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.387 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:10:07.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:10:07.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.76045237899999, 2024-09-13T09:10:07.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:10:07.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.760429249, 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:10:07.396 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:10:07.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:10:07.400 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.75781248600001, 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.401 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:10:07.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.75778840000001, 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:10:07.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:10:07.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.410 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.75776174699999, 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:10:07.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:10:07.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:07.415 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.75773810699999, 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:10:07.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:10:07.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.753922566, 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.420 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:10:07.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.424 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.753899023, 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.425 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:10:07.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.753875733, 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.431 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:10:07.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.434 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.75385222, 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:10:07.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.75379376399999, 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.439 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:10:07.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.443 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.749762828, 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.444 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:10:07.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:07.448 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 134.42986453099999, 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:10:07.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:07.453 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 134.42749275200001, 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:07.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:10:07.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 134.426958837, 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:10:07.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:10:07.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:10:07.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:10:07.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 134.42355831399999, 2024-09-13T09:10:07.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:10:07.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:10:07.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:07.471 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 134.42255850699999, 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:10:07.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:10:07.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 134.41938822899999, 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:10:07.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:10:07.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:07.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:07.481 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:07.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:10:07.483 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:07.480+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:07.584 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:07.580+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:07.654 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:07.654 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:07.654 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:07.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:10:07.713 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:07.712+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:07.785 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:07.784+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:07.838 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:07.838 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:07.838 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:07.839 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:07.839 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:07.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:10:07.887 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:07.881+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:07.988 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:07.984+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:08.018 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.575590987, 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.019 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:08.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.575566404, 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.024 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:10:08.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57554239999999, 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.029 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:10:08.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:10:08.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:10:08.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:10:08.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:10:08.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57551891700001, 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:10:08.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.036 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:08.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:10:08.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57549551, 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:10:08.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:10:08.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.044 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57547226099999, 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.045 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:10:08.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:10:08.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.575440155, 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:08.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:10:08.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.054 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57540453799999, 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.055 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.056 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.056 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.056 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:10:08.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.056 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:10:08.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.575372558, 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.060 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:10:08.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.064 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.575303279, 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.065 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:10:08.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:10:08.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.069 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57527821900001, 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:10:08.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:10:08.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57525478900001, 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:10:08.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.078 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.575231406, 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.079 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:10:08.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:08.083 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57520799, 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:10:08.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:08.088 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57518406700001, 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57515098299999, 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:10:08.093 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:10:08.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57512718300001, 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:08.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:10:08.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.57510285699999, 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.575070297, 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:10:08.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:10:08.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.112 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 152.575030424, 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:08.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:08.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:10:08.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:08.117 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:08.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:10:08.119 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:08.116+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:08.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:10:08.220 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:08.217+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:08.263 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:08.263 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:08.263 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:08.263 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:08.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:10:08.323 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:08.321+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:08.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:10:08.441 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:08.441 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:08.442 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:08.437+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:08.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:10:08.551 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:08.549+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:08.557 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:08.557 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:08.574 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:08.575 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:08.592 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:08.592 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:08.609 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:08.609 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:08.625 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:08.624+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:08.629 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:08.629 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:08.632 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:08.632 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:08.646 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:10:08.650 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:08.650 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:08.670 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:08.671 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:08.671 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:10:08.674 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:10:08.674 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:10:08.726 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:08.721+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:08.765 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:08.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:10:08.827 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:08.825+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:08.928 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:08.924+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:08.947 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:08.947 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:08.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:10:09.029 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:09.028+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:09.070 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:10:09.099 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:09.100 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:09.117 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:10:09.129 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:09.128+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:09.230 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:09.225+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:09.293 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:09.293 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:09.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:10:09.331 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:09.328+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:09.433 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:09.429+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:09.476 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:09.476 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:09.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:10:09.534 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:09.529+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:09.634 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:09.629+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:09.652 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:09.652 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:09.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:10:09.752 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:09.749+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:09.836 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:09.836 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:09.837 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:09.836+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:09.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:10:09.937 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:09.933+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:10.018 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:10.034 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:10:10.038 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:10.036+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:10.139 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:10.133+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:10.217 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:10.217 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:10.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:10:10.275 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:10.273+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:10.341 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:10.340+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:10.370 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:10.370 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:10.385 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:10:10.442 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:10.441+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:10.543 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:10.540+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:10.554 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:10.554 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:10.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:10:10.645 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:10.641+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:10.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:10:10.741 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:10.741 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:10.745 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:10.741+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:10.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:10:10.832 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:10.832 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:10.846 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:10.844+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:10.849 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:10.849 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:10.865 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:10.865 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:10.882 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:10.882 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:10.903 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:10.903 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:10.923 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:10.923 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:10.927 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:10.927 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:10.927 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:10.927 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:10.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:10:10.944 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:10.944 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:10.944 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:10:10.949 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:10:10.950 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:10:10.985 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:10.984+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:11.048 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:11.045+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:11.099 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:10:11.114 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:11.115 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:11.115 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:11.115 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:11.115 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:11.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:10:11.148 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:11.145+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:11.250 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:11.248+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:11.268 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:11.268 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:11.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:11.268 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:11.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.268 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18554 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10092544~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.268 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.64640379299999, 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10659373799999999, 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18554 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618113+0000", 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618112+0000", 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618115+0000", 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618124+0000", 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3070000000000003e-06 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618130+0000", 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4860000000000002e-06 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724337+0000", 2024-09-13T09:10:11.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106206565 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724536+0000", 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019917099999999999 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724706+0000", 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017070900000000001 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18555 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10141696~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.64638046300001, 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10690643499999999, 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18555 2024-09-13T09:10:11.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618136+0000", 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618135+0000", 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618138+0000", 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0060000000000001e-06 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618147+0000", 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618153+0000", 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.64e-06 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724772+0000", 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106618224 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.724894+0000", 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122106 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725042+0000", 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014886200000000001 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18556 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10174464~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.646357446, 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107292085, 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18556 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618159+0000", 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618158+0000", 2024-09-13T09:10:11.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618161+0000", 2024-09-13T09:10:11.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9629999999999998e-06 2024-09-13T09:10:11.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618170+0000", 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1999999999999994e-06 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618176+0000", 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5470000000000002e-06 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725139+0000", 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106962418 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725277+0000", 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013834200000000001 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725451+0000", 2024-09-13T09:10:11.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017422499999999999 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18557 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10256384~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.64633431600001, 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107586316, 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.284 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18557 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618182+0000", 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618181+0000", 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618184+0000", 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0199999999999999e-06 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618193+0000", 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1969999999999996e-06 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.618199+0000", 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5200000000000003e-06 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725507+0000", 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107307555 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725628+0000", 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121199 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.725769+0000", 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140579 2024-09-13T09:10:11.288 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18561 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10452992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.64371755299999, 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106177643, 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.289 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18561 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620799+0000", 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620798+0000", 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620802+0000", 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2930000000000001e-06 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620810+0000", 2024-09-13T09:10:11.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620817+0000", 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9070000000000001e-06 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726722+0000", 2024-09-13T09:10:11.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105904996 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726835+0000", 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112426 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.726977+0000", 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014192499999999999 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.293 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18562 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10485760~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.64369346699999, 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10654981099999999, 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18562 2024-09-13T09:10:11.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620823+0000", 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620822+0000", 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620825+0000", 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620837+0000", 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1806e-05 2024-09-13T09:10:11.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620844+0000", 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727059+0000", 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106214777 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727200+0000", 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014149900000000001 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727373+0000", 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017245199999999999 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18563 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10534912~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:11.298 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.643666814, 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106828123, 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18563 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:11.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620850+0000", 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620849+0000", 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620852+0000", 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0630000000000002e-06 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620860+0000", 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-06 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620867+0000", 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7830000000000001e-06 2024-09-13T09:10:11.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727426+0000", 2024-09-13T09:10:11.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10655848499999999 2024-09-13T09:10:11.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727535+0000", 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109499 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727678+0000", 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014276600000000001 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.303 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18564 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10567680~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.643643174, 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107093014, 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18564 2024-09-13T09:10:11.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620873+0000", 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620886+0000", 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620896+0000", 2024-09-13T09:10:11.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.620903+0000", 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9940000000000003e-06 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727720+0000", 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106817092 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727825+0000", 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010522300000000001 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.727966+0000", 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014144200000000001 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18572 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 10960896~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.308 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.63982763300001, 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10628952999999999, 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18572 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624689+0000", 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624688+0000", 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624691+0000", 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1939999999999998e-06 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624700+0000", 2024-09-13T09:10:11.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2400000000000007e-06 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624706+0000", 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7900000000000002e-06 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730700+0000", 2024-09-13T09:10:11.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105993658 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730837+0000", 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136396 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.730978+0000", 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014186900000000001 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18573 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11042816~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.63980409000001, 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106602267, 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18573 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624712+0000", 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624715+0000", 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1269999999999999e-06 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624723+0000", 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1669999999999999e-06 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624730+0000", 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7059999999999998e-06 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731032+0000", 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10630186599999999 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731168+0000", 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136272 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731315+0000", 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014673900000000001 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18574 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11075584~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.63978080000001, 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106968811, 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18574 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624736+0000", 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624735+0000", 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624738+0000", 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0730000000000001e-06 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624746+0000", 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1459999999999996e-06 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624753+0000", 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8600000000000004e-06 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731399+0000", 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10664582 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731538+0000", 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138966 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731705+0000", 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016654199999999999 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.323 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18575 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11124736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.324 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:11.324 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.63975728700001, 2024-09-13T09:10:11.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10724805599999999, 2024-09-13T09:10:11.324 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.324 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.324 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.325 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.325 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.325 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18575 2024-09-13T09:10:11.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.325 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624759+0000", 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624762+0000", 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0529999999999998e-06 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624770+0000", 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0059999999999992e-06 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624776+0000", 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731751+0000", 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106974678 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.731863+0000", 2024-09-13T09:10:11.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111853 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732007+0000", 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014440199999999999 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18576 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11157504~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.639698831, 2024-09-13T09:10:11.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10763399999999999, 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18576 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624818+0000", 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624817+0000", 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624844+0000", 2024-09-13T09:10:11.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7344000000000001e-05 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624862+0000", 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8323000000000001e-05 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.624887+0000", 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.404e-05 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732097+0000", 2024-09-13T09:10:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107209995 2024-09-13T09:10:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732264+0000", 2024-09-13T09:10:11.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016763300000000001 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.732452+0000", 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018754500000000001 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:18583 3.10 3:0895de43:::rbd_data.1124d0fab35.000000000000000c:c [read 11517952~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.63566789500001, 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106155318, 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.335 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 18583 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628849+0000", 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628848+0000", 2024-09-13T09:10:11.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628859+0000", 2024-09-13T09:10:11.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1446e-05 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628873+0000", 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3903999999999999e-05 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.628881+0000", 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7090000000000002e-06 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734691+0000", 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10581027699999999 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.734847+0000", 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155363 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:34.735004+0000", 2024-09-13T09:10:11.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015725900000000001 2024-09-13T09:10:11.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.344 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.344 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.344 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39023 3.1f 3:f9351101:::rbd_data.1124d4e7fcc5.0000000000000018:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:11.344 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:11.344 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 138.315769598, 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91536604899999996, 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39023 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948747+0000", 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948745+0000", 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948772+0000", 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6996e-05 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948809+0000", 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7079000000000002e-05 2024-09-13T09:10:11.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948823+0000", 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3873000000000001e-05 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948950+0000", 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126814 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.948976+0000", 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.599e-05 2024-09-13T09:10:11.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.863774+0000", 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91479874100000003 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.864113+0000", 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033866300000000001 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39044 3.1f 3:fec185c7:::rbd_data.1124d4e7fcc5.000000000000002d:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 138.31339781899999, 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91449410600000003, 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39044 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951119+0000", 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951117+0000", 2024-09-13T09:10:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951128+0000", 2024-09-13T09:10:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1007e-05 2024-09-13T09:10:11.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951157+0000", 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9006e-05 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951176+0000", 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8842999999999999e-05 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951285+0000", 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010911 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951308+0000", 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2986999999999999e-05 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865344+0000", 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91403623899999997 2024-09-13T09:10:11.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865613+0000", 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026843100000000002 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39048 3.1f 3:fe6e60e0:::rbd_data.1124d4e7fcc5.0000000000000031:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 138.31286390400001, 2024-09-13T09:10:11.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91426844699999998, 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39048 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951653+0000", 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951651+0000", 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:11.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951660+0000", 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6799999999999999e-06 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951681+0000", 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1345999999999999e-05 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951694+0000", 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3117e-05 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951829+0000", 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013501299999999999 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.951848+0000", 2024-09-13T09:10:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8335999999999999e-05 2024-09-13T09:10:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865684+0000", 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91383646399999996 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865921+0000", 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023701500000000001 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.361 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.362 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39072 3.1f 3:fb1e8e42:::rbd_data.1124d4e7fcc5.0000000000000049:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:11.362 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:11.362 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 138.309463381, 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.911154927, 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39072 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:11.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955053+0000", 2024-09-13T09:10:11.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955051+0000", 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955063+0000", 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.184e-05 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955095+0000", 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1942e-05 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955118+0000", 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2846000000000001e-05 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955256+0000", 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013791000000000001 2024-09-13T09:10:11.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.955281+0000", 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.865992+0000", 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91071113800000003 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866208+0000", 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215658 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39074 3.1f 3:f9de338b:::rbd_data.1124d4e7fcc5.000000000000004b:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 138.308463574, 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91040069199999996, 2024-09-13T09:10:11.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39074 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956053+0000", 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956045+0000", 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999924 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956062+0000", 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.664e-05 2024-09-13T09:10:11.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956088+0000", 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6492999999999999e-05 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956101+0000", 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.294e-05 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956201+0000", 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9611000000000006e-05 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.956217+0000", 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5999e-05 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866277+0000", 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91005970700000005 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866454+0000", 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017704600000000001 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:39080 3.1f 3:fa281cb4:::rbd_data.1124d4e7fcc5.0000000000000051:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 138.305293296, 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.907687456, 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.373 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 39080 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959223+0000", 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959221+0000", 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959233+0000", 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.166e-05 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959264+0000", 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0799000000000001e-05 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959278+0000", 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.435e-05 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959374+0000", 2024-09-13T09:10:11.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6100999999999994e-05 2024-09-13T09:10:11.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:11.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:52.959387+0000", 2024-09-13T09:10:11.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2856e-05 2024-09-13T09:10:11.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866611+0000", 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90722398400000004 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:53.866911+0000", 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029968300000000002 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.378 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.379 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:11.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:10:11.381 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:11.377+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:11.481 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:11.480+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:11.554 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:11.554 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:11.554 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:11.554 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:11.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:10:11.609 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:11.608+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:11.683 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:11.680+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:11.735 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:11.735 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:11.736 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:11.736 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:11.736 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:11.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:10:11.793 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:11.792+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:11.885 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:11.881+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:11.923 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:11.923 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:11.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:11.923 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:11.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.480351695, 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.924 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:10:11.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.928 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.480327112, 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:10:11.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48030310799999, 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:10:11.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.937 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48027962500001, 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:10:11.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.942 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48025621799999, 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:11.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:10:11.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48023296900001, 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.947 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:11.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:10:11.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48020086299999, 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:11.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48016524600001, 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.957 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.961 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.480133266, 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:11.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:10:11.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48006398699999, 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:10:11.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:10:11.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48003892700001, 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:10:11.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.48001549700001, 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:10:11.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.479992114, 2024-09-13T09:10:11.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.47996869799999, 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.985 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:11.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:10:11.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.989 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.47994477500001, 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:10:11.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:10:11.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.47991169100001, 2024-09-13T09:10:11.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:11.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:10:11.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.479887891, 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:11.999 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:12.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:10:12.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.47986356499999, 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.004 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:12.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:10:12.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:12.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:12.008 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.47983100499999, 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:12.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:10:12.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:10:12.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 156.479791132, 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:12.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:12.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:12.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:12.017 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:12.018 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:12.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:10:12.019 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:12.017+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:12.081 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 1, 7] out_osds: [0] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-13T09:10:12.081 INFO:tasks.thrashosds.thrasher:Reviving osd 7 2024-09-13T09:10:12.081 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T09:10:12.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T09:10:12.083 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:10:12.085 INFO:tasks.ceph.osd.7:Started 2024-09-13T09:10:12.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:10:12.119 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:12.117+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:12.165 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:12.165 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:12.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:10:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-13T09:10:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-13T09:10:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-13T09:10:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-13T09:10:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-13T09:10:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-13T09:10:12.215 INFO:teuthology.orchestra.run.smithi060.stderr:config diff 2024-09-13T09:10:12.216 INFO:teuthology.orchestra.run.smithi060.stderr:config diff get 2024-09-13T09:10:12.216 INFO:teuthology.orchestra.run.smithi060.stderr:config get 2024-09-13T09:10:12.216 INFO:teuthology.orchestra.run.smithi060.stderr:config help [] 2024-09-13T09:10:12.216 INFO:teuthology.orchestra.run.smithi060.stderr:config set ... 2024-09-13T09:10:12.216 INFO:teuthology.orchestra.run.smithi060.stderr:admin_socket: invalid command 2024-09-13T09:10:12.217 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T09:10:12.218 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2024-09-13T09:10:12.223 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:12.221+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:12.322 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:12.317+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:12.346 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:12.346 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:12.346 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:12.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:10:12.448 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:12.445+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:12.524 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:12.521+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:12.529 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:12.529 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:12.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41498767799999, 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41496691099999, 2024-09-13T09:10:12.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:12.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:10:12.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41494905900001, 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:10:12.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41493249300001, 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:10:12.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:10:12.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:10:12.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:12.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41490690800001, 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:12.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:10:12.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:12.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.414887741, 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:10:12.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.414870458, 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:12.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:10:12.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41434158800001, 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:10:12.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:10:12.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.414309683, 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:10:12.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:10:12.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.414284735, 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:10:12.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:10:12.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.575 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41398017, 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:10:12.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.412217747, 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:12.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.584 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.412196897, 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:10:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:10:12.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41217457299999, 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:10:12.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:10:12.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41215588399999, 2024-09-13T09:10:12.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41213883200001, 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:12.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:10:12.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.41212180799999, 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.602 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:10:12.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:12.606 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 132.40210882100001, 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:12.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:10:12.610 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.021954297999997, 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.611 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:10:12.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.021773875999997, 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:12.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:12.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:10:12.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.620 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:12.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:12.621 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:12.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:10:12.664 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:12.661+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:12.690 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:12.688+0000 7fe53c9aa7c0 -1 Falling back to public interface 2024-09-13T09:10:12.725 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:12.724+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:12.794 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:12.794 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:12.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:10:12.826 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:12.824+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:12.927 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:12.921+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:12.935 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:12.935 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:12.935 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:12.935 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:12.935 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:12.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:10:12.950 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:10:13.035 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:13.033+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:13.128 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:13.125+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154208 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.029201925999999, 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019831428, 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154208 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110898+0000", 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110909+0000", 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0896999999999999e-05 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110932+0000", 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3544999999999999e-05 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110943+0000", 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.064e-05 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119027+0000", 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080840670000000003 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119033+0000", 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0050000000000001e-06 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130694+0000", 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116608949999999 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130732+0000", 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7696e-05 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154209 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.029143869000002, 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199906329999999, 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154209 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110961+0000", 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110967+0000", 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4470000000000003e-06 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110972+0000", 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119044+0000", 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080713150000000008 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119046+0000", 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130938+0000", 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0118917460000001 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130949+0000", 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0607e-05 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154218 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.029067988999998, 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019944518, 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154218 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:13.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111033+0000", 2024-09-13T09:10:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111038+0000", 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111047+0000", 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1779999999999992e-06 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111053+0000", 2024-09-13T09:10:13.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9089999999999997e-06 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119060+0000", 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080075760000000006 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119065+0000", 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130971+0000", 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119057490000001 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130979+0000", 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8790000000000003e-06 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154219 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.029026209000001, 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199227849999999, 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154219 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111075+0000", 2024-09-13T09:10:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111079+0000", 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111086+0000", 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111090+0000", 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119079+0000", 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007988396 2024-09-13T09:10:13.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119081+0000", 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130995+0000", 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011913643 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130999+0000", 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6940000000000001e-06 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154228 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.028685852999999, 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019595799, 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154228 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111415+0000", 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111421+0000", 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111432+0000", 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1807000000000001e-05 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111439+0000", 2024-09-13T09:10:13.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8909999999999998e-06 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119092+0000", 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076528159999999998 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119097+0000", 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:10:13.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131008+0000", 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011911539 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131012+0000", 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5760000000000002e-06 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154229 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.027920238, 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01884818, 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154229 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112181+0000", 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112186+0000", 2024-09-13T09:10:13.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249999999999997e-06 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112202+0000", 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.543e-05 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112209+0000", 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5669999999999997e-06 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119115+0000", 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069053999999999999 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119120+0000", 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131023+0000", 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011903446 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131030+0000", 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154238 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.027885927, 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188343550000001, 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154238 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112219+0000", 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0240000000000002e-06 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112225+0000", 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112230+0000", 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119130+0000", 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068996450000000003 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119132+0000", 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131046+0000", 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119135260000001 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131050+0000", 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5859999999999998e-06 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154239 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.027867452999999, 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188390510000001, 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154239 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112234+0000", 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112237+0000", 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112243+0000", 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112247+0000", 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119149+0000", 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069020679999999999 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119154+0000", 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131067+0000", 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011912659 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131074+0000", 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9709999999999998e-06 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154242 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.027848050999999, 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188413620000001, 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154242 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112256+0000", 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112262+0000", 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112267+0000", 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119166+0000", 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068991599999999997 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119168+0000", 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.678e-06 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131088+0000", 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011919435 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131095+0000", 2024-09-13T09:10:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5769999999999996e-06 2024-09-13T09:10:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154243 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.027820198999997, 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018836657, 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154243 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112281+0000", 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112285+0000", 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112292+0000", 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.126e-06 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112297+0000", 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0760000000000002e-06 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119178+0000", 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068802890000000004 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119180+0000", 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131111+0000", 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011930947 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131119+0000", 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154244 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.027800051, 2024-09-13T09:10:13.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188321789999999, 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154244 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:13.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112304+0000", 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112310+0000", 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1859999999999997e-06 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112315+0000", 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119189+0000", 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068746950000000001 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119192+0000", 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131128+0000", 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01193613 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131134+0000", 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0240000000000003e-06 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154251 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.027201660999999, 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018254725, 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154251 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112899+0000", 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112905+0000", 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5709999999999997e-06 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112916+0000", 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112923+0000", 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119201+0000", 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062780379999999997 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119204+0000", 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131151+0000", 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011947269 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131155+0000", 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.284e-06 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154252 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.026060237000003, 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171325529999999, 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.205 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154252 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114041+0000", 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114046+0000", 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114062+0000", 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5951999999999999e-05 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114070+0000", 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9510000000000006e-06 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119213+0000", 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051428860000000002 2024-09-13T09:10:13.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119215+0000", 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4509999999999999e-06 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131171+0000", 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119552249999999 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131174+0000", 2024-09-13T09:10:13.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6739999999999999e-06 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154253 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.026025027000003, 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017116229, 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.210 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154253 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114080+0000", 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114087+0000", 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1679999999999998e-06 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114092+0000", 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119225+0000", 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051325920000000001 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119227+0000", 2024-09-13T09:10:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:10:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131187+0000", 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011959702 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131193+0000", 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154260 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.025988267999999, 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017101802, 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154260 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114113+0000", 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114117+0000", 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3139999999999999e-06 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114125+0000", 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1429999999999998e-06 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114131+0000", 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:10:13.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119237+0000", 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051056859999999999 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119239+0000", 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131212+0000", 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119728100000001 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131216+0000", 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7160000000000001e-06 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154271 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.025965812999999, 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017099328, 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154271 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:13.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114138+0000", 2024-09-13T09:10:13.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114145+0000", 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114150+0000", 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119251+0000", 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051011809999999998 2024-09-13T09:10:13.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119256+0000", 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131232+0000", 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011976298 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131236+0000", 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6030000000000001e-06 2024-09-13T09:10:13.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154272 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.025948133999997, 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017104572, 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.227 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154272 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114156+0000", 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114162+0000", 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114167+0000", 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.977e-06 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119274+0000", 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051065360000000001 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119279+0000", 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:10:13.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131252+0000", 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011972874 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131259+0000", 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154273 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.025930289999998, 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171116979999999, 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154273 2024-09-13T09:10:13.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114171+0000", 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114174+0000", 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114181+0000", 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5030000000000002e-06 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114185+0000", 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6720000000000003e-06 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119297+0000", 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051116549999999997 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119302+0000", 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131276+0000", 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011973451 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131283+0000", 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.627e-06 2024-09-13T09:10:13.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154274 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.025913989999999, 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171203449999999, 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154274 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114190+0000", 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7519999999999999e-06 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114193+0000", 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:10:13.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114195+0000", 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.407e-06 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119320+0000", 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051245739999999998 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119323+0000", 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-13T09:10:13.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131301+0000", 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01197782 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131308+0000", 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5440000000000001e-06 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154293 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.022885686999999, 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0141231429999999, 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154293 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:13.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117215+0000", 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:13.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117223+0000", 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117236+0000", 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3492000000000001e-05 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:13.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117247+0000", 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0725e-05 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119334+0000", 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002087392 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119337+0000", 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131332+0000", 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119952649999999 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131340+0000", 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2089999999999999e-06 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:13.246 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:13.247 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:13.248 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:13.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:10:13.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:13.249 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:13.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:13.250 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:13.250 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:13.250 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:13.251 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:13.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:13.251 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:13.251 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:13.244+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:13.264 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:13.264 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:13.264 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:10:13.268 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:10:13.269 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:10:13.348 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:13.348+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:13.379 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:13.380 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:13.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:10:13.449 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:13.445+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:13.518 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:13.519 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:13.533 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:10:13.576 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:13.573+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:13.650 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:13.645+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:13.700 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:13.700 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:13.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:10:13.756 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:13.753+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:13.852 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:13.849+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:13.881 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:13.881 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:13.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:10:13.988 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:13.985+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:14.053 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:14.052+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:14.069 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:14.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:10:14.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:10:14.154 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:14.152+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:14.255 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:14.249+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:14.271 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:14.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:10:14.356 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:14.353+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:14.419 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:14.419 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:14.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:10:14.458 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:14.453+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:14.558 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:14.556+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:14.560 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:14.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:10:14.659 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:14.657+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:14.744 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:14.744 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:14.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:10:14.760 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:14.757+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:14.862 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:14.857+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:14.885 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:14.886 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:14.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:10:14.963 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:14.960+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:15.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:10:15.064 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:15.061+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:15.125 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:10:15.164 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:15.160+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:15.217 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:15.218 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:15.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:10:15.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:10:15.281 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:15.280+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:15.366 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:15.364+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:15.412 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:15.412 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:15.423 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:15.423 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:15.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:10:15.439 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:15.440 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:15.456 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:15.456 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:15.466 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:15.461+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:15.473 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:15.473 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:15.490 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:15.490 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:15.507 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:15.507 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:15.526 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:15.526 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:15.526 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:10:15.533 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:10:15.533 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:10:15.568 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:15.564+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:15.589 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:15.589 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:15.589 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:15.589 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:15.589 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:15.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:10:15.668 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:15.665+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:15.769 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:15.769 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21521 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13713408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32758605999999, 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103133351, 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.770 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21521 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439602+0000", 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439601+0000", 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439604+0000", 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.084e-06 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439613+0000", 2024-09-13T09:10:15.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9230000000000008e-06 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439620+0000", 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.72e-06 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542307+0000", 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10268674999999999 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542523+0000", 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021602599999999999 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542735+0000", 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212578 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21522 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13795328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.774 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32756147699999, 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10348389500000001, 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21522 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439627+0000", 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439626+0000", 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439629+0000", 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2839999999999998e-06 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439638+0000", 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7099999999999996e-06 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439645+0000", 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7599999999999997e-06 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542842+0000", 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103197346 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.542980+0000", 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000138152 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543110+0000", 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130283 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21523 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13828096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32753747300001, 2024-09-13T09:10:15.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103969853, 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21523 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439651+0000", 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439650+0000", 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439653+0000", 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9170000000000002e-06 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439662+0000", 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439668+0000", 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6470000000000001e-06 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543205+0000", 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10353648999999999 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543427+0000", 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000222573 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543620+0000", 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019286999999999999 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21524 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13877248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:15.783 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32751399, 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10441205200000001, 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21524 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:15.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439674+0000", 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439673+0000", 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439676+0000", 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9660000000000001e-06 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439685+0000", 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7770000000000008e-06 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439691+0000", 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4269999999999996e-06 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543696+0000", 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104004374 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.543897+0000", 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020086999999999999 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544086+0000", 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018930100000000001 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.787 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21525 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13910016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32749058300001, 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.104881744, 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21525 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.788 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439697+0000", 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439700+0000", 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0469999999999998e-06 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439709+0000", 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7560000000000005e-06 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439715+0000", 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5969999999999997e-06 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544158+0000", 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10444318299999999 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544358+0000", 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000199836 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544579+0000", 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000220972 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21526 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 13991936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.327467334, 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105225551, 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21526 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439721+0000", 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439720+0000", 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439723+0000", 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.027e-06 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439732+0000", 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7029999999999995e-06 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439747+0000", 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4786e-05 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544673+0000", 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10492623299999999 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544815+0000", 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014265600000000001 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.544946+0000", 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130779 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21527 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14024704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32743522800001, 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105704908, 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21527 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439753+0000", 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439752+0000", 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439766+0000", 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3986e-05 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439775+0000", 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1200000000000008e-06 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439782+0000", 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9399999999999996e-06 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545035+0000", 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105253337 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545250+0000", 2024-09-13T09:10:15.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021444899999999999 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545458+0000", 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020771200000000001 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21528 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14073856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.327399611, 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106172273, 2024-09-13T09:10:15.800 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21528 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439788+0000", 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439798+0000", 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042e-05 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439808+0000", 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4069999999999999e-06 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439814+0000", 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8499999999999996e-06 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545536+0000", 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.1057215 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545751+0000", 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021526499999999999 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.545961+0000", 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000209478 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21529 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14106624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:15.804 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32736763099999, 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106629523, 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21529 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439820+0000", 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439823+0000", 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.236e-06 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439832+0000", 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7970000000000006e-06 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439838+0000", 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999999e-06 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546033+0000", 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10619446 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546245+0000", 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021249499999999999 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546450+0000", 2024-09-13T09:10:15.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204568 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21530 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14188544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32729835200001, 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106936685, 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.809 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21530 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439890+0000", 2024-09-13T09:10:15.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439889+0000", 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439892+0000", 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4630000000000001e-06 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439902+0000", 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4700000000000008e-06 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439908+0000", 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8229999999999997e-06 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546549+0000", 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106640231 2024-09-13T09:10:15.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546697+0000", 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014811199999999999 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546826+0000", 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129479 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21531 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14221312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.327273292, 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107429041, 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21531 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439915+0000", 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439914+0000", 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439917+0000", 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2229999999999999e-06 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439926+0000", 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5469999999999997e-06 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439932+0000", 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4069999999999998e-06 2024-09-13T09:10:15.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.546923+0000", 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.106990654 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547135+0000", 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021242099999999999 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547344+0000", 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208419 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21532 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14270464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.327249862, 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.107712289, 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.818 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21532 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439938+0000", 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439937+0000", 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439941+0000", 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3730000000000001e-06 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439949+0000", 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8400000000000001e-06 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439956+0000", 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3439999999999997e-06 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547420+0000", 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10746464 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547535+0000", 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114172 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547650+0000", 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011578000000000001 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21533 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14303232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32722647899999, 2024-09-13T09:10:15.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108197949, 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21533 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439962+0000", 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439961+0000", 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439964+0000", 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1030000000000002e-06 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439973+0000", 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439979+0000", 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4230000000000002e-06 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547734+0000", 2024-09-13T09:10:15.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10775462199999999 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.547953+0000", 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021921399999999999 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548159+0000", 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020641299999999999 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21534 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14385152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32720306300001, 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108516186, 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21534 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.827 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439985+0000", 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439984+0000", 2024-09-13T09:10:15.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439987+0000", 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1970000000000001e-06 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.439996+0000", 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8659999999999995e-06 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440003+0000", 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548233+0000", 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.108230092 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548351+0000", 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117989 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548501+0000", 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015007200000000001 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21535 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14417920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32717914, 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109015536, 2024-09-13T09:10:15.831 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21535 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440009+0000", 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440008+0000", 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440011+0000", 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.926e-06 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440020+0000", 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7569999999999993e-06 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440036+0000", 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5627000000000001e-05 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548593+0000", 2024-09-13T09:10:15.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10855701500000001 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.548814+0000", 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022187199999999999 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549024+0000", 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020997499999999999 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21536 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14467072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.327146056, 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109289806, 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21536 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440042+0000", 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440041+0000", 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440044+0000", 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.18e-06 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440053+0000", 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9770000000000006e-06 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440060+0000", 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4760000000000003e-06 2024-09-13T09:10:15.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549108+0000", 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109047768 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549218+0000", 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549332+0000", 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113572 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21537 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14499840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.327122256, 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109773546, 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.840 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21537 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440066+0000", 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440065+0000", 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440068+0000", 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9239999999999999e-06 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440077+0000", 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7299999999999994e-06 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440084+0000", 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7630000000000003e-06 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549406+0000", 2024-09-13T09:10:15.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.109322616 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549631+0000", 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022446799999999999 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549839+0000", 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000208679 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21538 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14581760~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32709793000001, 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11006544, 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21538 2024-09-13T09:10:15.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440090+0000", 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440089+0000", 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440092+0000", 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0089999999999999e-06 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440101+0000", 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7539999999999995e-06 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440116+0000", 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5339e-05 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.549917+0000", 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10980013600000001 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550045+0000", 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012842600000000001 2024-09-13T09:10:15.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550155+0000", 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110432 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21539 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14614528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32706537000001, 2024-09-13T09:10:15.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11053217, 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21539 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440123+0000", 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440122+0000", 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440140+0000", 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8150000000000001e-05 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440149+0000", 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3700000000000001e-06 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440156+0000", 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8360000000000003e-06 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550233+0000", 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110076351 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550447+0000", 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021431799999999999 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550655+0000", 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000207832 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:21540 3.12 3:49e8dbee:::rbd_data.1124d0fab35.0000000000000009:c [read 14663680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:15.853 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 160.32702549699999, 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11079618099999999, 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 21540 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440162+0000", 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440171+0000", 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3540000000000006e-06 2024-09-13T09:10:15.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440180+0000", 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3000000000000007e-06 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.440191+0000", 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.011e-05 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550736+0000", 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11054576000000001 2024-09-13T09:10:15.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550849+0000", 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011270200000000001 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:07:35.550959+0000", 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109639 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:15.858 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:15.858 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:15.858 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:15.765+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:15.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:10:15.870 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:15.868+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:15.971 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:15.968+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:15.993 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:15.993 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:15.993 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:15.993 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:16.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:10:16.071 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:16.069+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:16.134 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:16.134 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:16.149 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:10:16.172 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:16.168+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:16.272 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:16.272+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:16.329 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:16.329 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21467175800001, 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:10:16.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:10:16.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.334 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21465099100001, 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:16.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:10:16.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.214633139, 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.214616573, 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:10:16.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.214590988, 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:10:16.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:10:16.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21457182099999, 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:10:16.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21455453799999, 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:10:16.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:10:16.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.214025668, 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.362 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:10:16.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21399376299999, 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:10:16.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:10:16.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21396881499999, 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21366424999999, 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.375 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:10:16.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21190182699999, 2024-09-13T09:10:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21188097699999, 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:10:16.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21185865300001, 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21183996400001, 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:10:16.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:10:16.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.211822912, 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:10:16.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.21180588799999, 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:10:16.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:10:16.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:16.406 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 136.201792901, 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:10:16.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:10:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.821638378000003, 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:10:16.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:10:16.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 94.821457956000003, 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:10:16.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:10:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:10:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:10:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:10:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:10:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:10:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.423 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:16.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:10:16.426 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:16.424+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:16.526 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:16.521+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:16.553 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:16.553 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:16.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:10:16.609 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:16.608+0000 7fe53c9aa7c0 -1 osd.7 157 log_to_monitors true 2024-09-13T09:10:16.626 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:16.624+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:16.727 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:16.721+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:16.733 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:16.733 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:16.733 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:16.733 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:16.733 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:16.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:10:16.840 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:16.837+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:16.922 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154208 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.808409971000003, 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019831428, 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154208 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:16.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110898+0000", 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110909+0000", 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0896999999999999e-05 2024-09-13T09:10:16.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110932+0000", 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3544999999999999e-05 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110943+0000", 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.064e-05 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119027+0000", 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080840670000000003 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119033+0000", 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0050000000000001e-06 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130694+0000", 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116608949999999 2024-09-13T09:10:16.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130732+0000", 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7696e-05 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154209 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:16.928 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.808351913999999, 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199906329999999, 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154209 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110961+0000", 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110967+0000", 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4470000000000003e-06 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110972+0000", 2024-09-13T09:10:16.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119044+0000", 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080713150000000008 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119046+0000", 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130938+0000", 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0118917460000001 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130949+0000", 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0607e-05 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154218 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.808276034000002, 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019944518, 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154218 2024-09-13T09:10:16.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111033+0000", 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111038+0000", 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111047+0000", 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1779999999999992e-06 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111053+0000", 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9089999999999997e-06 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119060+0000", 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080075760000000006 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119065+0000", 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130971+0000", 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119057490000001 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130979+0000", 2024-09-13T09:10:16.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8790000000000003e-06 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154219 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.808234253999998, 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199227849999999, 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154219 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:16.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111075+0000", 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111079+0000", 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111086+0000", 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111090+0000", 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119079+0000", 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007988396 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119081+0000", 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130995+0000", 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011913643 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130999+0000", 2024-09-13T09:10:16.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6940000000000001e-06 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154228 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.807893898000003, 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019595799, 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154228 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111415+0000", 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111421+0000", 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111432+0000", 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1807000000000001e-05 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111439+0000", 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8909999999999998e-06 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119092+0000", 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076528159999999998 2024-09-13T09:10:16.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119097+0000", 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131008+0000", 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011911539 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131012+0000", 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5760000000000002e-06 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154229 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.807128282999997, 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01884818, 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154229 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112181+0000", 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112186+0000", 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249999999999997e-06 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112202+0000", 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.543e-05 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112209+0000", 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5669999999999997e-06 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119115+0000", 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069053999999999999 2024-09-13T09:10:16.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119120+0000", 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131023+0000", 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011903446 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131030+0000", 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154238 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.807093971999997, 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188343550000001, 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154238 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112219+0000", 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0240000000000002e-06 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112225+0000", 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112230+0000", 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119130+0000", 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068996450000000003 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119132+0000", 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131046+0000", 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119135260000001 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131050+0000", 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5859999999999998e-06 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154239 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.807075498000003, 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188390510000001, 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154239 2024-09-13T09:10:16.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112234+0000", 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112237+0000", 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112243+0000", 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112247+0000", 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119149+0000", 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069020679999999999 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119154+0000", 2024-09-13T09:10:16.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131067+0000", 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011912659 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131074+0000", 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9709999999999998e-06 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154242 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.807056095999997, 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188413620000001, 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154242 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112256+0000", 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112262+0000", 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112267+0000", 2024-09-13T09:10:16.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119166+0000", 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068991599999999997 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119168+0000", 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.678e-06 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131088+0000", 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011919435 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131095+0000", 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5769999999999996e-06 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154243 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.807028244000001, 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018836657, 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154243 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112281+0000", 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112285+0000", 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112292+0000", 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.126e-06 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112297+0000", 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0760000000000002e-06 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119178+0000", 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068802890000000004 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119180+0000", 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131111+0000", 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011930947 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131119+0000", 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154244 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.807008095999997, 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188321789999999, 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154244 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112304+0000", 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112310+0000", 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1859999999999997e-06 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112315+0000", 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119189+0000", 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068746950000000001 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119192+0000", 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:10:16.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131128+0000", 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01193613 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131134+0000", 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0240000000000003e-06 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.978 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154251 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.806409705999997, 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018254725, 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154251 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112899+0000", 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:16.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112905+0000", 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5709999999999997e-06 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112916+0000", 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112923+0000", 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119201+0000", 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062780379999999997 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119204+0000", 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131151+0000", 2024-09-13T09:10:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011947269 2024-09-13T09:10:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131155+0000", 2024-09-13T09:10:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.284e-06 2024-09-13T09:10:16.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154252 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.805268282, 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171325529999999, 2024-09-13T09:10:16.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154252 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114041+0000", 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114046+0000", 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114062+0000", 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5951999999999999e-05 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114070+0000", 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9510000000000006e-06 2024-09-13T09:10:16.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119213+0000", 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051428860000000002 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119215+0000", 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4509999999999999e-06 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131171+0000", 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119552249999999 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131174+0000", 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6739999999999999e-06 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.989 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154253 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.805233072, 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017116229, 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154253 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114080+0000", 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114087+0000", 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1679999999999998e-06 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114092+0000", 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119225+0000", 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051325920000000001 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119227+0000", 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131187+0000", 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011959702 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131193+0000", 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154260 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.805196313000003, 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017101802, 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154260 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:16.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114113+0000", 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114117+0000", 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3139999999999999e-06 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114125+0000", 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1429999999999998e-06 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114131+0000", 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119237+0000", 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051056859999999999 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119239+0000", 2024-09-13T09:10:16.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131212+0000", 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119728100000001 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131216+0000", 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7160000000000001e-06 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:16.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154271 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.805173858000003, 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017099328, 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154271 2024-09-13T09:10:17.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114138+0000", 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114145+0000", 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114150+0000", 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119251+0000", 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051011809999999998 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119256+0000", 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-13T09:10:17.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131232+0000", 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011976298 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131236+0000", 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6030000000000001e-06 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154272 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.805156179000001, 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017104572, 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154272 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114156+0000", 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114162+0000", 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114167+0000", 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.977e-06 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119274+0000", 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051065360000000001 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119279+0000", 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131252+0000", 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011972874 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131259+0000", 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.009 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154273 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.805138335000002, 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171116979999999, 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154273 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114171+0000", 2024-09-13T09:10:17.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114174+0000", 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114181+0000", 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5030000000000002e-06 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114185+0000", 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6720000000000003e-06 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119297+0000", 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051116549999999997 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119302+0000", 2024-09-13T09:10:17.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131276+0000", 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011973451 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131283+0000", 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.627e-06 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.014 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154274 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.805122034999997, 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171203449999999, 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154274 2024-09-13T09:10:17.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114190+0000", 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7519999999999999e-06 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114193+0000", 2024-09-13T09:10:17.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114195+0000", 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.407e-06 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119320+0000", 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051245739999999998 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119323+0000", 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131301+0000", 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01197782 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131308+0000", 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5440000000000001e-06 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154293 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.802093732000003, 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0141231429999999, 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154293 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117215+0000", 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117223+0000", 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117236+0000", 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3492000000000001e-05 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117247+0000", 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0725e-05 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119334+0000", 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002087392 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119337+0000", 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131332+0000", 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119952649999999 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131340+0000", 2024-09-13T09:10:17.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2089999999999999e-06 2024-09-13T09:10:17.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.025 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:17.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:10:17.027 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:17.021+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:17.128 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:17.125+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:17.152 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:10:17.205 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:17.204+0000 7fe530771640 -1 osd.7 157 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T09:10:17.213 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:17.213 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:17.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:10:17.228 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:17.225+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:17.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:10:17.329 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:17.328+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:17.360 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:17.360 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:17.360 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:17.360 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:17.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:10:17.398 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:17.398 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:17.398 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:17.398 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:17.398 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:17.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:10:17.456 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:17.453+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:17.531 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:17.529+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:17.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:10:17.580 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23895234, 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:17.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:10:17.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.585 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23891503499999, 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.586 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.238895321, 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.590 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:10:17.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:10:17.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23887899600001, 2024-09-13T09:10:17.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:17.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:10:17.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23886313, 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.599 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:10:17.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:10:17.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23884734500001, 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:10:17.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.607 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23882827400001, 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.608 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:10:17.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23881172700001, 2024-09-13T09:10:17.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:10:17.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.616 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23878412799999, 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:10:17.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:17.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23876688600001, 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:10:17.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23873644299999, 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.626 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.238717528, 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.630 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23869973999999, 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:10:17.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.238674516, 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.639 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:10:17.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.643 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23865631199999, 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:10:17.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:10:17.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:10:17.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:17.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23863599399999, 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:17.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:10:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23861744600001, 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.653 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:10:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.657 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.23860017199999, 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:10:17.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:10:17.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 162.238582718, 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.662 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:10:17.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:162508 3.b 3.9f9e59eb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-13T09:10:17.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 39.170040954000001, 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.47181327899999997, 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 162508 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:17.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407494+0000", 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407503+0000", 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0699999999999996e-06 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407523+0000", 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9692999999999999e-05 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407535+0000", 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2689e-05 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407598+0000", 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2064999999999996e-05 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407606+0000", 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9560000000000003e-06 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.879280+0000", 2024-09-13T09:10:17.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.47167379599999998 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.879309+0000", 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8436999999999999e-05 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:17.671 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:17.672 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:17.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:10:17.673 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:17.674 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:17.674 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:17.672+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:17.687 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:17.687 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:17.703 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:17.703 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:17.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:17.719 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:17.740 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:17.740 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:17.760 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:17.760 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:17.774 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:17.772+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:17.780 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:17.780 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:17.780 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-13T09:10:17.784 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-13T09:10:17.784 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-13T09:10:17.805 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:17.805 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:17.819 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:10:17.875 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:17.872+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:17.976 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:17.972+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:17.986 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:17.986 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:18.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:10:18.077 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:18.073+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:18.126 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:18.126 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:18.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:10:18.185 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:10:18.185 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:10:18.185 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:18.184+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:18.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:10:18.279 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:18.276+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:18.280 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:18.281 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:18.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:10:18.380 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:18.377+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:18.462 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:18.462 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:18.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:10:18.525 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:18.524+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:18.583 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:18.577+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:18.597 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:18.597 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:18.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:10:18.683 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:18.681+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:18.785 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:18.784+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:18.785 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:18.785 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:18.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:10:18.886 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:18.881+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:18.965 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:18.965 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:18.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:10:19.025 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:19.024+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:19.087 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:19.084+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:19.107 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:19.107 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:19.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:10:19.189 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:19.185+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:19.192 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:10:19.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:10:19.290 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:19.288+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:19.339 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:19.339 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:19.354 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:10:19.397 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:19.396+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:19.491 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:19.488+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:19.517 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:19.517 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:19.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:10:19.591 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:19.589+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:19.692 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:19.689+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:19.706 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:19.706 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:19.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:10:19.785 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:10:19.832 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:19.829+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:19.840 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:19.840 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:19.841 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:19.841 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:19.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:10:19.901 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:19.900+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:19.942 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:19.942 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:19.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:19.975 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:19.975 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:19.976 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-13T09:10:19.976 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-13T09:10:19.976 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:19.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:10:19.991 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:19.992 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:19.995 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:19.989+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:20.096 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:20.093+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:20.165 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:20.165 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167427 3.3 3:c446c987:::rbd_header.1124d0fab35:head [watch reconnect cookie 140193707146800 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:18.187619+0000", 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.972962557, 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.020624867, 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.166 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167427 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187619+0000", 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187619+0000", 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.167 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187617+0000", 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187627+0000", 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0718e-05 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187645+0000", 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7992e-05 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.168 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187660+0000", 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4204e-05 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187809+0000", 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000149796 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.169 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187835+0000", 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5763e-05 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.200067+0000", 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.012231543 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.200262+0000", 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019552400000000001 2024-09-13T09:10:20.170 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.204879+0000", 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.004616963 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for rw locks", 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.204980+0000", 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010107699999999999 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.171 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.205654+0000", 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00067409399999999997 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.205773+0000", 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118923 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.208244+0000", 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002470547 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.172 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167428 3.3 3:c446c987:::rbd_header.1124d0fab35:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e181)", 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:18.187683+0000", 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.9728986040000001, 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.020782933, 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167428 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.173 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187683+0000", 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187683+0000", 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187682+0000", 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.174 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187698+0000", 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5671999999999998e-05 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187709+0000", 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1042e-05 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.175 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187727+0000", 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7618999999999999e-05 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187916+0000", 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018936100000000001 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:18.187930+0000", 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4452e-05 2024-09-13T09:10:20.176 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.200554+0000", 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.012623812 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.200628+0000", 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3751999999999995e-05 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.177 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.205008+0000", 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043803000000000002 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for rw locks", 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.205092+0000", 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4153000000000006e-05 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.206341+0000", 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001248133 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.178 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.206413+0000", 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2953000000000004e-05 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.207083+0000", 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0006692 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.208259+0000", 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001176243 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.179 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.208303+0000", 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4486000000000001e-05 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.208350+0000", 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6872000000000001e-05 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.180 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.208466+0000", 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011556399999999999 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167432 3.3 3:c446c987:::rbd_header.1124d0fab35:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.181 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.212547+0000", 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.94803443200000004, 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025181119999999999, 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167432 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.182 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.212547+0000", 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.212547+0000", 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.212546+0000", 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.183 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.212554+0000", 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4530000000000002e-06 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.212571+0000", 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7050000000000001e-05 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.184 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.212581+0000", 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.023e-05 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.212658+0000", 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6765000000000001e-05 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.185 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.212772+0000", 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011341900000000001 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.213059+0000", 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000287531 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.214191+0000", 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011316080000000001 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.186 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.214942+0000", 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000750949 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.214997+0000", 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4894e-05 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215065+0000", 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8561999999999998e-05 2024-09-13T09:10:20.187 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167434 3.3 3:c446c987:::rbd_header.1124d0fab35:head [watch unwatch cookie 140193707146800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.215315+0000", 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.94526665200000004, 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002271871, 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.188 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167434 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215315+0000", 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215315+0000", 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.189 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215314+0000", 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215318+0000", 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1930000000000003e-06 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.190 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215327+0000", 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8780000000000004e-06 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215334+0000", 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7159999999999997e-06 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215694+0000", 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035968099999999999 2024-09-13T09:10:20.191 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215859+0000", 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000165432 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216150+0000", 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029082400000000002 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.192 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.217413+0000", 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012626779999999999 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.217432+0000", 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9406e-05 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.217475+0000", 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3210999999999997e-05 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.193 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.217587+0000", 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011146600000000001 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167435 3.3 3:c446c987:::rbd_header.1124d0fab35:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.215806+0000", 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.94477577999999995, 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00206814, 2024-09-13T09:10:20.194 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167435 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215806+0000", 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.195 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215806+0000", 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215804+0000", 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.196 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215813+0000", 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8599999999999999e-06 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215828+0000", 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5024999999999999e-05 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.215854+0000", 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5876000000000001e-05 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216940+0000", 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001085646 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for rw locks", 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.217042+0000", 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000102002 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.217621+0000", 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00057882599999999995 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.217708+0000", 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7406999999999998e-05 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.217874+0000", 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016580800000000001 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGRecoveryDelete(3.8 e182,181 3:16ca5fe4:::rbd_data.11245864225e.0000000000000000:head,162'846)", 2024-09-13T09:10:20.199 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.216447+0000", 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.94413407100000002, 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0072858489999999996, 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216447+0000", 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216447+0000", 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216447+0000", 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.201 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216453+0000", 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8029999999999999e-06 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216468+0000", 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5064e-05 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216488+0000", 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9701e-05 2024-09-13T09:10:20.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216598+0000", 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010954399999999999 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.216604+0000", 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8299999999999998e-06 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.223733+0000", 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0071288289999999997 2024-09-13T09:10:20.203 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGRecoveryDelete(3.8 e182,181 3:1792e69c:::rbd_data.11241b071601.0000000000000002:head,167'848)", 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.230644+0000", 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.92993790899999995, 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028025519999999998, 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.230644+0000", 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.230644+0000", 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.230643+0000", 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.230649+0000", 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7479999999999999e-06 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.230660+0000", 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1797000000000001e-05 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.230669+0000", 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3119999999999994e-06 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.230940+0000", 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000271198 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.230945+0000", 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1189999999999996e-06 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.233446+0000", 2024-09-13T09:10:20.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025011909999999998 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167449 3.3 3:c446c987:::rbd_header.1124d0fab35:head [watch watch cookie 140193910815024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.236189+0000", 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.92439199699999997, 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019780430000000001, 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.208 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167449 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236189+0000", 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236189+0000", 2024-09-13T09:10:20.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236189+0000", 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236193+0000", 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1620000000000001e-06 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236203+0000", 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5789999999999993e-06 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236208+0000", 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2120000000000002e-06 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236260+0000", 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2065999999999999e-05 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236360+0000", 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9920999999999994e-05 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.236556+0000", 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019635200000000001 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.237922+0000", 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0013662629999999999 2024-09-13T09:10:20.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.237980+0000", 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7169000000000001e-05 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.238034+0000", 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4104999999999998e-05 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.238168+0000", 2024-09-13T09:10:20.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013388399999999999 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167451 3.3 3:c446c987:::rbd_header.1124d0fab35:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.239039+0000", 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.92154242399999997, 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0040448599999999999, 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.214 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167451 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.239039+0000", 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.239039+0000", 2024-09-13T09:10:20.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.239038+0000", 2024-09-13T09:10:20.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:20.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.239047+0000", 2024-09-13T09:10:20.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4380000000000001e-06 2024-09-13T09:10:20.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.239065+0000", 2024-09-13T09:10:20.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7592e-05 2024-09-13T09:10:20.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.239075+0000", 2024-09-13T09:10:20.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0152e-05 2024-09-13T09:10:20.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.239773+0000", 2024-09-13T09:10:20.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069801399999999995 2024-09-13T09:10:20.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.239891+0000", 2024-09-13T09:10:20.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011832699999999999 2024-09-13T09:10:20.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.240069+0000", 2024-09-13T09:10:20.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017788100000000001 2024-09-13T09:10:20.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.242875+0000", 2024-09-13T09:10:20.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002805554 2024-09-13T09:10:20.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.242939+0000", 2024-09-13T09:10:20.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4152000000000005e-05 2024-09-13T09:10:20.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.243024+0000", 2024-09-13T09:10:20.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5025999999999995e-05 2024-09-13T09:10:20.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.243084+0000", 2024-09-13T09:10:20.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0044999999999999e-05 2024-09-13T09:10:20.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.225 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.226 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.4388.0:167451 3.3 e182/181 ondisk, result = 0)", 2024-09-13T09:10:20.226 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.241042+0000", 2024-09-13T09:10:20.226 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.91953992600000001, 2024-09-13T09:10:20.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020516039999999998, 2024-09-13T09:10:20.227 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.227 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:20.227 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.241042+0000", 2024-09-13T09:10:20.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.241042+0000", 2024-09-13T09:10:20.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.241041+0000", 2024-09-13T09:10:20.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:20.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.241046+0000", 2024-09-13T09:10:20.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7770000000000002e-06 2024-09-13T09:10:20.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.241055+0000", 2024-09-13T09:10:20.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1370000000000008e-06 2024-09-13T09:10:20.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.241061+0000", 2024-09-13T09:10:20.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9769999999999997e-06 2024-09-13T09:10:20.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.242919+0000", 2024-09-13T09:10:20.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001858375 2024-09-13T09:10:20.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.242930+0000", 2024-09-13T09:10:20.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1209999999999999e-05 2024-09-13T09:10:20.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.243093+0000", 2024-09-13T09:10:20.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016285200000000001 2024-09-13T09:10:20.235 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.235 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.235 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.236 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167460 3.3 3:c446c987:::rbd_header.1124d0fab35:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.236 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.250576+0000", 2024-09-13T09:10:20.236 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.91000578200000004, 2024-09-13T09:10:20.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024143260000000001, 2024-09-13T09:10:20.237 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.237 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.237 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167460 2024-09-13T09:10:20.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.238 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250576+0000", 2024-09-13T09:10:20.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250576+0000", 2024-09-13T09:10:20.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250575+0000", 2024-09-13T09:10:20.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:20.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250580+0000", 2024-09-13T09:10:20.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4110000000000004e-06 2024-09-13T09:10:20.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250590+0000", 2024-09-13T09:10:20.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4930000000000004e-06 2024-09-13T09:10:20.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250596+0000", 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1630000000000001e-06 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250634+0000", 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8612000000000002e-05 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250701+0000", 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6383999999999993e-05 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.250821+0000", 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011975499999999999 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251258+0000", 2024-09-13T09:10:20.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000437799 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.252936+0000", 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016771069999999999 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.252967+0000", 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1619000000000003e-05 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.252990+0000", 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2895e-05 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(osd.4.0:3808 3.8 e182/181 3:165d178b:::rbd_data.1124cd332510.0000000000000002:d v 182'866, mlcod=175'863)", 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.251118+0000", 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.90946386400000001, 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00302426, 2024-09-13T09:10:20.246 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251118+0000", 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251118+0000", 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251117+0000", 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251124+0000", 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251139+0000", 2024-09-13T09:10:20.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4844999999999999e-05 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251146+0000", 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6179999999999997e-06 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251269+0000", 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000122905 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.251290+0000", 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0638000000000001e-05 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254102+0000", 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028119590000000002 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254142+0000", 2024-09-13T09:10:20.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9932000000000003e-05 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167462 3.3 3:c446c987:::rbd_header.1124d0fab35:head [watch unwatch cookie 140193910815024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.253296+0000", 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.90728581600000002, 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030486739999999999, 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.251 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167462 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.253296+0000", 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.253296+0000", 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.253295+0000", 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.253299+0000", 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3170000000000002e-06 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.253305+0000", 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5429999999999999e-06 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.253310+0000", 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.198e-06 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254041+0000", 2024-09-13T09:10:20.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00073086899999999998 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254132+0000", 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0598000000000002e-05 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254245+0000", 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011268 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.255568+0000", 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001323262 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.256168+0000", 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00060025699999999998 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.256225+0000", 2024-09-13T09:10:20.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7129000000000001e-05 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.256344+0000", 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119202 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167463 3.3 3:c446c987:::rbd_header.1124d0fab35:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.254068+0000", 2024-09-13T09:10:20.257 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.90651322099999998, 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026191109999999999, 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167463 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254068+0000", 2024-09-13T09:10:20.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254068+0000", 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254067+0000", 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254074+0000", 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3230000000000003e-06 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254087+0000", 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3774999999999999e-05 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.254096+0000", 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9970000000000004e-06 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.255335+0000", 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001238835 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for rw locks", 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.255439+0000", 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103516 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.256485+0000", 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010465069999999999 2024-09-13T09:10:20.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.256578+0000", 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2555999999999999e-05 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.256687+0000", 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109619 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.262 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(osd.4.0:3810 3.8 e182/181 3:111865e7:::rbd_data.112498ca9569.000000000000002f:20 v 182'870, mlcod=182'866)", 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.266237+0000", 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.89434412200000002, 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024895210000000002, 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.266237+0000", 2024-09-13T09:10:20.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.266237+0000", 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.266237+0000", 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.266242+0000", 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4330000000000003e-06 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.266249+0000", 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.266256+0000", 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4739999999999999e-06 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.266297+0000", 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0694e-05 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.266314+0000", 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7045999999999999e-05 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.268679+0000", 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002365097 2024-09-13T09:10:20.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.268727+0000", 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7732000000000003e-05 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167486 3.3 3:c446c987:::rbd_header.1124d0fab35:head [watch unwatch cookie 140193705406352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.281542+0000", 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.87903969999999998, 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018054119999999999, 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.267 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167486 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281542+0000", 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281542+0000", 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281541+0000", 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281545+0000", 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5719999999999999e-06 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281553+0000", 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4540000000000007e-06 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281559+0000", 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9789999999999998e-06 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281695+0000", 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000135363 2024-09-13T09:10:20.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281760+0000", 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5152000000000002e-05 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281918+0000", 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015856099999999999 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.282500+0000", 2024-09-13T09:10:20.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058216599999999996 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.283208+0000", 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00070748299999999996 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.283252+0000", 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4467e-05 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.283347+0000", 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4797999999999996e-05 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167487 3.3 3:c446c987:::rbd_header.1124d0fab35:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.281862+0000", 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.87871983499999995, 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001695298, 2024-09-13T09:10:20.273 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167487 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281862+0000", 2024-09-13T09:10:20.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281862+0000", 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281861+0000", 2024-09-13T09:10:20.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281867+0000", 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0800000000000002e-06 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281879+0000", 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2703e-05 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.281889+0000", 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.004e-05 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.282315+0000", 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00042545799999999998 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for rw locks", 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.282407+0000", 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2408e-05 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.283392+0000", 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00098515600000000005 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.283458+0000", 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5895999999999994e-05 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.283557+0000", 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8587999999999999e-05 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.279 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167491 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_state_set in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.289439+0000", 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.87114264299999999, 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017395679999999999, 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167491 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.280 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289439+0000", 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289439+0000", 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289437+0000", 2024-09-13T09:10:20.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:20.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289446+0000", 2024-09-13T09:10:20.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1710000000000002e-06 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289461+0000", 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5444e-05 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289470+0000", 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8559999999999996e-06 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289652+0000", 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000182182 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289753+0000", 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100466 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.289982+0000", 2024-09-13T09:10:20.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000229461 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.290838+0000", 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00085559799999999995 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.291072+0000", 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023438300000000001 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.291130+0000", 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7938e-05 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.291178+0000", 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8486999999999998e-05 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.4388.0:167520 3.8 e182/181 3:17a17bcd:::rbd_data.1124d0fab35.0000000000000002:head v 182'871, mlcod=182'868)", 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.315886+0000", 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.84469503300000004, 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0079784299999999999, 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.315886+0000", 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.315886+0000", 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.315885+0000", 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.999999 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.315895+0000", 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2469999999999997e-06 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.315915+0000", 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0247000000000001e-05 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.315923+0000", 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9610000000000005e-06 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.315978+0000", 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5671000000000003e-05 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.316001+0000", 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3067000000000001e-05 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.323834+0000", 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0078320569999999999 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.323865+0000", 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1338999999999999e-05 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.4388.0:167531 3.c 3:3264b9b7:::rbd_data.1124d0fab35.000000000000000d:head [delete] snapc c=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-13T09:10:19.316907+0000", 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.843674445, 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001935052, 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.4388", 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 167531 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.292 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.316907+0000", 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.316907+0000", 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.316905+0000", 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.316913+0000", 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9130000000000003e-06 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.316929+0000", 2024-09-13T09:10:20.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.556e-05 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.316937+0000", 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7060000000000004e-06 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-13T09:10:20.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.316981+0000", 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4311000000000003e-05 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.317158+0000", 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017703700000000001 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3", 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.317280+0000", 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121839 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.318101+0000", 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00082072799999999995 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:20.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.318764+0000", 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00066308199999999995 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.318802+0000", 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8185000000000001e-05 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-13T09:10:19.318842+0000", 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0281999999999998e-05 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-13T09:10:20.299 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-13T09:10:20.299 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:20.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:10:20.300 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:10:20.300 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:20.301 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:20.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:10:20.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:10:20.303 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:20.303 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:20.303 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:20.304 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:20.304 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:20.297+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:20.308 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:10:20.308 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_pg_delete_cost = '' (not observed, change may require restart) osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-13T09:10:20.401 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:20.397+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:20.441 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:20.442 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:20.460 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:10:20.502 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:20.500+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:20.602 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:20.597+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:20.622 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:20.622 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:20.622 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:20.622 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:20.622 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:20.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:10:20.703 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:20.700+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:20.804 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:20.801+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:20.806 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:20.806 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.69049815899999, 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:10:20.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:20.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:10:20.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.812 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.69047739199999, 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.813 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:10:20.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:20.817 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.69045954000001, 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:10:20.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:10:20.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.69044297400001, 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:20.826 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.690417389, 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:20.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:10:20.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:10:20.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.690398222, 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:20.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.835 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.69038093899999, 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:10:20.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:20.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.68985206900001, 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:10:20.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.689820164, 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.68979521599999, 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:10:20.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:10:20.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:10:20.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.689490651, 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.855 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:20.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:10:20.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:10:20.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.687728228, 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:10:20.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:10:20.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:10:20.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:10:20.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:10:20.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:10:20.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.687707378, 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.867 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:10:20.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.68768505400001, 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.872 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.68766636500001, 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:10:20.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:20.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:10:20.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:10:20.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.68764931300001, 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:20.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:10:20.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:10:20.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.68763228899999, 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:20.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 140.67761930200001, 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:10:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.297464778999995, 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:10:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:10:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 99.297284356999995, 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:10:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:10:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:10:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:20.910 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:20.912 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:10:20.912 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:20.909+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:21.012 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:21.008+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:21.085 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:21.085 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:21.086 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:21.086 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:21.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:10:21.143 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:21.141+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:21.213 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:21.209+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:21.223 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:21.223 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:21.223 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:21.224 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:21.224 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:21.224 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:10:21.227 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:10:21.251 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:10:21.262 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:10:21.313 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:21.312+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:21.382 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154208 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.268798292, 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019831428, 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154208 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110898+0000", 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110909+0000", 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0896999999999999e-05 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110932+0000", 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3544999999999999e-05 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110943+0000", 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.064e-05 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119027+0000", 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080840670000000003 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119033+0000", 2024-09-13T09:10:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0050000000000001e-06 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130694+0000", 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116608949999999 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130732+0000", 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7696e-05 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154209 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.268740235000003, 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199906329999999, 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154209 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110961+0000", 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110967+0000", 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4470000000000003e-06 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110972+0000", 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:10:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119044+0000", 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080713150000000008 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119046+0000", 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130938+0000", 2024-09-13T09:10:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0118917460000001 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130949+0000", 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0607e-05 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154218 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.268664354999999, 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019944518, 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154218 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111033+0000", 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111038+0000", 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111047+0000", 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1779999999999992e-06 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111053+0000", 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9089999999999997e-06 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119060+0000", 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080075760000000006 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119065+0000", 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130971+0000", 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119057490000001 2024-09-13T09:10:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130979+0000", 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8790000000000003e-06 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154219 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.268622575000002, 2024-09-13T09:10:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199227849999999, 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154219 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111075+0000", 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111079+0000", 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:10:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111086+0000", 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111090+0000", 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119079+0000", 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007988396 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119081+0000", 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130995+0000", 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011913643 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130999+0000", 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6940000000000001e-06 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154228 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.268282219, 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019595799, 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154228 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111415+0000", 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111421+0000", 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111432+0000", 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1807000000000001e-05 2024-09-13T09:10:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111439+0000", 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8909999999999998e-06 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119092+0000", 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076528159999999998 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119097+0000", 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131008+0000", 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011911539 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131012+0000", 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5760000000000002e-06 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154229 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.267516604000001, 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01884818, 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154229 2024-09-13T09:10:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112181+0000", 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112186+0000", 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249999999999997e-06 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112202+0000", 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.543e-05 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112209+0000", 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5669999999999997e-06 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119115+0000", 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069053999999999999 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119120+0000", 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-13T09:10:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131023+0000", 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011903446 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131030+0000", 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154238 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.267482293, 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188343550000001, 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154238 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112219+0000", 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0240000000000002e-06 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112225+0000", 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112230+0000", 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119130+0000", 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068996450000000003 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119132+0000", 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:10:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131046+0000", 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119135260000001 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131050+0000", 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5859999999999998e-06 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154239 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.267463819, 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188390510000001, 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154239 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112234+0000", 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112237+0000", 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112243+0000", 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112247+0000", 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119149+0000", 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069020679999999999 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119154+0000", 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131067+0000", 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011912659 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131074+0000", 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9709999999999998e-06 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154242 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.267444417, 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188413620000001, 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154242 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112256+0000", 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112262+0000", 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112267+0000", 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119166+0000", 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068991599999999997 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119168+0000", 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.678e-06 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131088+0000", 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011919435 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131095+0000", 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5769999999999996e-06 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154243 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.267416564999998, 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018836657, 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154243 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112281+0000", 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112285+0000", 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112292+0000", 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.126e-06 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112297+0000", 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0760000000000002e-06 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119178+0000", 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068802890000000004 2024-09-13T09:10:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119180+0000", 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131111+0000", 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011930947 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131119+0000", 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-09-13T09:10:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154244 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.267396417, 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188321789999999, 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154244 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112304+0000", 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112310+0000", 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1859999999999997e-06 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112315+0000", 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119189+0000", 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068746950000000001 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119192+0000", 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131128+0000", 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01193613 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131134+0000", 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0240000000000003e-06 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154251 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.266798027, 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018254725, 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154251 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112899+0000", 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112905+0000", 2024-09-13T09:10:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5709999999999997e-06 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112916+0000", 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112923+0000", 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119201+0000", 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062780379999999997 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119204+0000", 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131151+0000", 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011947269 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131155+0000", 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.284e-06 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154252 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.265656602999996, 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171325529999999, 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154252 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114041+0000", 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114046+0000", 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114062+0000", 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5951999999999999e-05 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114070+0000", 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9510000000000006e-06 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119213+0000", 2024-09-13T09:10:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051428860000000002 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119215+0000", 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4509999999999999e-06 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131171+0000", 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119552249999999 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131174+0000", 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6739999999999999e-06 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154253 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.265621393000004, 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017116229, 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154253 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114080+0000", 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:10:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114087+0000", 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1679999999999998e-06 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114092+0000", 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119225+0000", 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051325920000000001 2024-09-13T09:10:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119227+0000", 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131187+0000", 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011959702 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131193+0000", 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154260 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.265584634, 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017101802, 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154260 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114113+0000", 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114117+0000", 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3139999999999999e-06 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114125+0000", 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1429999999999998e-06 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114131+0000", 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119237+0000", 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051056859999999999 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119239+0000", 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131212+0000", 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119728100000001 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131216+0000", 2024-09-13T09:10:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7160000000000001e-06 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154271 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.265562179, 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017099328, 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154271 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114138+0000", 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114145+0000", 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114150+0000", 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119251+0000", 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051011809999999998 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119256+0000", 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131232+0000", 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011976298 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131236+0000", 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6030000000000001e-06 2024-09-13T09:10:21.458 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154272 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.265544499999997, 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017104572, 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.459 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154272 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114156+0000", 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114162+0000", 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114167+0000", 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.977e-06 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119274+0000", 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051065360000000001 2024-09-13T09:10:21.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119279+0000", 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131252+0000", 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011972874 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131259+0000", 2024-09-13T09:10:21.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154273 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.265526655999999, 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171116979999999, 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154273 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114171+0000", 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114174+0000", 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:10:21.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114181+0000", 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5030000000000002e-06 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114185+0000", 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6720000000000003e-06 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119297+0000", 2024-09-13T09:10:21.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051116549999999997 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119302+0000", 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131276+0000", 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011973451 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131283+0000", 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.627e-06 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154274 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.265510356, 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171203449999999, 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154274 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:21.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114190+0000", 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7519999999999999e-06 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114193+0000", 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114195+0000", 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.407e-06 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119320+0000", 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051245739999999998 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119323+0000", 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131301+0000", 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01197782 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131308+0000", 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5440000000000001e-06 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154293 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 62.262482052999999, 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0141231429999999, 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154293 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:21.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117215+0000", 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:21.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117223+0000", 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117236+0000", 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3492000000000001e-05 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117247+0000", 2024-09-13T09:10:21.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0725e-05 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119334+0000", 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002087392 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119337+0000", 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131332+0000", 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119952649999999 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131340+0000", 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2089999999999999e-06 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:21.478 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:21.480 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:10:21.480 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:21.477+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:21.581 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:21.577+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:21.639 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:21.639 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:21.640 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:21.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:10:21.681 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:21.680+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:21.782 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:21.777+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:21.820 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:21.820 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:21.821 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:21.821 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:21.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:10:21.883 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:21.880+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:21.985 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:21.981+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:22.010 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.668547878, 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:10:22.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:10:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:10:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66851057299999, 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:10:22.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.020 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.668490859, 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:10:22.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66847453400001, 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.025 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:10:22.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.668458668, 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:10:22.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:22.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66844288300001, 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:22.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:10:22.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66842381199999, 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.043 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66840726500001, 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:10:22.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66837966599999, 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:10:22.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:10:22.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66836242400001, 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:22.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:10:22.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66833198099999, 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:10:22.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:10:22.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.668313066, 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.062 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:10:22.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66829527799999, 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:10:22.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.668270054, 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.071 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66825184999999, 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.075 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:10:22.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.079 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66823153199999, 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:22.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:10:22.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.084 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66821298400001, 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.66819570999999, 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:10:22.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:10:22.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:10:22.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 166.668178256, 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:10:22.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:162508 3.b 3.9f9e59eb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 43.599636492000002, 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.47181327899999997, 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 162508 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407494+0000", 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407503+0000", 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0699999999999996e-06 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407523+0000", 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9692999999999999e-05 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407535+0000", 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2689e-05 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407598+0000", 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2064999999999996e-05 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:22.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407606+0000", 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9560000000000003e-06 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.879280+0000", 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.47167379599999998 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.879309+0000", 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8436999999999999e-05 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:22.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:22.104 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:22.105 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:10:22.106 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:22.104+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:22.206 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:22.204+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:22.222 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:10:22.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:10:22.290 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:22.290 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:22.291 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:22.291 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:22.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:10:22.307 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:22.304+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:22.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:10:22.407 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:22.405+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:22.481 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:22.481 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:22.481 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:22.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:10:22.499 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:22.500 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:22.509 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:22.508+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:22.516 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:22.516 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:22.533 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:22.533 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:22.549 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:22.549 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:22.584 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:22.584 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:22.610 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:22.608+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:22.621 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:22.621 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:22.636 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:22.636 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:22.640 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:22.641 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:22.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:10:22.666 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:10:22.666 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:10:22.711 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:22.708+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:22.789 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:22.789 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:22.804 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:10:22.845 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:22.844+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:22.913 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:22.909+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:22.973 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:22.973 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:22.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:10:23.014 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:23.009+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:23.106 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:23.106 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:23.115 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:23.109+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:23.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:10:23.216 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:23.213+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:23.237 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:10:23.244 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:23.244 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:23.258 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T09:10:23.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:10:23.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:10:23.316 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:23.312+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:23.398 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:23.399 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:23.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:10:23.457 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:23.456+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:23.518 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:23.516+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:23.581 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:23.581 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:23.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:10:23.620 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:23.617+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:23.721 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:23.717+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:23.756 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:23.757 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:23.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:10:23.822 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:23.820+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:23.894 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:23.894 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:23.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:10:23.953 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:23.952+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:24.024 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:24.024+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:24.078 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:24.078 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:24.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:10:24.125 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:24.121+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:24.226 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:24.224+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:24.257 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-09-13T09:10:24.258 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:10:24.271 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:24.272 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:24.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:10:24.294 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:10:24.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:10:24.337 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:24.336+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:24.427 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:24.421+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:24.470 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:24.470 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:24.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:10:24.528 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:24.525+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:24.610 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:24.611 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:24.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T09:10:24.628 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:24.628+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:24.637 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:10:24.654 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-13T09:10:24.245010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117768549,"num_objects":553,"num_object_clones":57,"num_object_copies":1106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553,"num_whiteouts":388,"num_read":209340,"num_read_kb":689226,"num_write":24879,"num_write_kb":340443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2612,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21081,"ondisk_log_size":21081,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":431184,"kb_used_data":81072,"kb_used_omap":29,"kb_used_meta":349986,"kb_avail":660171696,"statfs":{"total":676457349120,"available":676015816704,"internally_reserved":0,"allocated":83017728,"data_stored":89094211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29807,"internal_metadata":358386577},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-359692785,"num_objects":-177,"num_object_clones":-71,"num_object_copies":-354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-177,"num_whiteouts":-66,"num_read":1028,"num_read_kb":885,"num_write":167,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.056010"},"pg_stats":[{"pgid":"3.1f","version":"184'565","reported_seq":5686,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.241242+0000","last_change":"2024-09-13T09:10:23.241545+0000","last_active":"2024-09-13T09:10:23.241242+0000","last_peered":"2024-09-13T09:10:23.241242+0000","last_clean":"2024-09-13T09:10:23.241242+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:23.241242+0000","last_undegraded":"2024-09-13T09:10:23.241242+0000","last_fullsized":"2024-09-13T09:10:23.241242+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073648230000000004,"stat_sum":{"num_bytes":2006600,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":5968,"num_read_kb":20217,"num_write":673,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"3.1e","version":"186'1202","reported_seq":4985,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.240432+0000","last_change":"2024-09-13T09:10:23.241104+0000","last_active":"2024-09-13T09:10:23.240432+0000","last_peered":"2024-09-13T09:10:23.240432+0000","last_clean":"2024-09-13T09:10:23.240432+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:10:23.240432+0000","last_undegraded":"2024-09-13T09:10:23.240432+0000","last_fullsized":"2024-09-13T09:10:23.240432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006939181,"stat_sum":{"num_bytes":262144,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":9083,"num_read_kb":19020,"num_write":1553,"num_write_kb":6026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"3.1d","version":"184'1399","reported_seq":5363,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.239763+0000","last_change":"2024-09-13T09:10:21.240487+0000","last_active":"2024-09-13T09:10:21.239763+0000","last_peered":"2024-09-13T09:10:21.239763+0000","last_clean":"2024-09-13T09:10:21.239763+0000","last_became_active":"2024-09-13T09:08:38.208191+0000","last_became_peered":"2024-09-13T09:08:38.208191+0000","last_unstale":"2024-09-13T09:10:21.239763+0000","last_undegraded":"2024-09-13T09:10:21.239763+0000","last_fullsized":"2024-09-13T09:10:21.239763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1399,"log_dups_size":0,"ondisk_log_size":1399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:04.686300+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.02041157,"stat_sum":{"num_bytes":10466292,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":18,"num_read":8733,"num_read_kb":34149,"num_write":2230,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.1c","version":"183'722","reported_seq":4054,"reported_epoch":184,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-13T09:10:21.229270+0000","last_change":"2024-09-13T09:10:21.229253+0000","last_active":"2024-09-13T09:10:21.229270+0000","last_peered":"2024-09-13T09:10:21.229270+0000","last_clean":"2024-09-13T09:10:21.229270+0000","last_became_active":"2024-09-13T09:08:38.209451+0000","last_became_peered":"2024-09-13T09:08:38.209451+0000","last_unstale":"2024-09-13T09:10:21.229270+0000","last_undegraded":"2024-09-13T09:10:21.229270+0000","last_fullsized":"2024-09-13T09:10:21.229270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:59.672566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022080989999999998,"stat_sum":{"num_bytes":1048576,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":5703,"num_read_kb":35589,"num_write":957,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.1b","version":"183'710","reported_seq":22123,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.343459+0000","last_change":"2024-09-13T09:10:20.208684+0000","last_active":"2024-09-13T09:10:20.343459+0000","last_peered":"2024-09-13T09:10:20.343459+0000","last_clean":"2024-09-13T09:10:20.343459+0000","last_became_active":"2024-09-13T09:08:38.210333+0000","last_became_peered":"2024-09-13T09:08:38.210333+0000","last_unstale":"2024-09-13T09:10:20.343459+0000","last_undegraded":"2024-09-13T09:10:20.343459+0000","last_fullsized":"2024-09-13T09:10:20.343459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:53.493070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026289769999999998,"stat_sum":{"num_bytes":2299681,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":23645,"num_read_kb":48156,"num_write":772,"num_write_kb":18569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.1a","version":"183'698","reported_seq":6432,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.230930+0000","last_change":"2024-09-13T09:10:21.230930+0000","last_active":"2024-09-13T09:10:21.230930+0000","last_peered":"2024-09-13T09:10:21.230930+0000","last_clean":"2024-09-13T09:10:21.230930+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:10:21.230930+0000","last_undegraded":"2024-09-13T09:10:21.230930+0000","last_fullsized":"2024-09-13T09:10:21.230930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017851049999999999,"stat_sum":{"num_bytes":2004071,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":8408,"num_read_kb":25820,"num_write":846,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.19","version":"178'384","reported_seq":2057,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.206126+0000","last_change":"2024-09-13T09:10:20.207203+0000","last_active":"2024-09-13T09:10:20.206126+0000","last_peered":"2024-09-13T09:10:20.206126+0000","last_clean":"2024-09-13T09:10:20.206126+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:10:20.206126+0000","last_undegraded":"2024-09-13T09:10:20.206126+0000","last_fullsized":"2024-09-13T09:10:20.206126+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012737,"stat_sum":{"num_bytes":450789,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":15,"num_read":1827,"num_read_kb":8140,"num_write":347,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.18","version":"184'711","reported_seq":3531,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.241687+0000","last_change":"2024-09-13T09:10:23.243272+0000","last_active":"2024-09-13T09:10:23.241687+0000","last_peered":"2024-09-13T09:10:23.241687+0000","last_clean":"2024-09-13T09:10:23.241687+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:10:23.241687+0000","last_undegraded":"2024-09-13T09:10:23.241687+0000","last_fullsized":"2024-09-13T09:10:23.241687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088804250000000008,"stat_sum":{"num_bytes":1042388,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":4805,"num_read_kb":14792,"num_write":762,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"3.17","version":"184'644","reported_seq":5828,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.242688+0000","last_change":"2024-09-13T09:10:23.243259+0000","last_active":"2024-09-13T09:10:23.242688+0000","last_peered":"2024-09-13T09:10:23.242688+0000","last_clean":"2024-09-13T09:10:23.242688+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:10:23.242688+0000","last_undegraded":"2024-09-13T09:10:23.242688+0000","last_fullsized":"2024-09-13T09:10:23.242688+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087551869999999993,"stat_sum":{"num_bytes":675672,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":7182,"num_read_kb":31245,"num_write":619,"num_write_kb":18265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"3.16","version":"184'700","reported_seq":2055,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.241499+0000","last_change":"2024-09-13T09:10:21.242266+0000","last_active":"2024-09-13T09:10:21.241499+0000","last_peered":"2024-09-13T09:10:21.241499+0000","last_clean":"2024-09-13T09:10:21.241499+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:21.241499+0000","last_undegraded":"2024-09-13T09:10:21.241499+0000","last_fullsized":"2024-09-13T09:10:21.241499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016871398999999999,"stat_sum":{"num_bytes":4537583,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":10,"num_read":2905,"num_read_kb":14193,"num_write":951,"num_write_kb":9806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.15","version":"178'695","reported_seq":4229,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-13T09:10:18.186196+0000","last_change":"2024-09-13T09:10:18.186196+0000","last_active":"2024-09-13T09:10:18.186196+0000","last_peered":"2024-09-13T09:10:18.186196+0000","last_clean":"2024-09-13T09:10:18.186196+0000","last_became_active":"2024-09-13T09:08:38.221484+0000","last_became_peered":"2024-09-13T09:08:38.221484+0000","last_unstale":"2024-09-13T09:10:18.186196+0000","last_undegraded":"2024-09-13T09:10:18.186196+0000","last_fullsized":"2024-09-13T09:10:18.186196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:04.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013267699999999999,"stat_sum":{"num_bytes":13390952,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":5093,"num_read_kb":24615,"num_write":673,"num_write_kb":16161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"16"}]},{"pgid":"3.14","version":"184'698","reported_seq":19349,"reported_epoch":184,"state":"active+clean+snaptrim","last_fresh":"2024-09-13T09:10:21.241925+0000","last_change":"2024-09-13T09:10:21.230427+0000","last_active":"2024-09-13T09:10:21.241925+0000","last_peered":"2024-09-13T09:10:21.241925+0000","last_clean":"2024-09-13T09:10:21.241925+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:10:21.241925+0000","last_undegraded":"2024-09-13T09:10:21.241925+0000","last_fullsized":"2024-09-13T09:10:21.241925+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.001800718,"stat_sum":{"num_bytes":12416133,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":21094,"num_read_kb":34154,"num_write":614,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.216771+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:10:20.216771+0000","last_peered":"2024-09-13T09:10:20.216771+0000","last_clean":"2024-09-13T09:10:20.216771+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:10:20.216771+0000","last_undegraded":"2024-09-13T09:10:20.216771+0000","last_fullsized":"2024-09-13T09:10:20.216771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"183'690","reported_seq":5937,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.209148+0000","last_change":"2024-09-13T09:10:20.209148+0000","last_active":"2024-09-13T09:10:20.209148+0000","last_peered":"2024-09-13T09:10:20.209148+0000","last_clean":"2024-09-13T09:10:20.209148+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:10:20.209148+0000","last_undegraded":"2024-09-13T09:10:20.209148+0000","last_fullsized":"2024-09-13T09:10:20.209148+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:13.066567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030828689999999998,"stat_sum":{"num_bytes":1048576,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":6169,"num_read_kb":21942,"num_write":788,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"2.6","version":"0'0","reported_seq":187,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-13T09:10:19.195120+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:10:19.195120+0000","last_peered":"2024-09-13T09:10:19.195120+0000","last_clean":"2024-09-13T09:10:19.195120+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:10:19.195120+0000","last_undegraded":"2024-09-13T09:10:19.195120+0000","last_fullsized":"2024-09-13T09:10:19.195120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"182'424","reported_seq":2475,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.208303+0000","last_change":"2024-09-13T09:10:20.208303+0000","last_active":"2024-09-13T09:10:20.208303+0000","last_peered":"2024-09-13T09:10:20.208303+0000","last_clean":"2024-09-13T09:10:20.208303+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:10:20.208303+0000","last_undegraded":"2024-09-13T09:10:20.208303+0000","last_fullsized":"2024-09-13T09:10:20.208303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023577979999999999,"stat_sum":{"num_bytes":1750594,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":17,"num_read":2724,"num_read_kb":14643,"num_write":385,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"2.5","version":"0'0","reported_seq":180,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.201671+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:10:20.201671+0000","last_peered":"2024-09-13T09:10:20.201671+0000","last_clean":"2024-09-13T09:10:20.201671+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:10:20.201671+0000","last_undegraded":"2024-09-13T09:10:20.201671+0000","last_fullsized":"2024-09-13T09:10:20.201671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"180'606","reported_seq":6165,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-13T09:10:18.185727+0000","last_change":"2024-09-13T09:10:18.185727+0000","last_active":"2024-09-13T09:10:18.185727+0000","last_peered":"2024-09-13T09:10:18.185727+0000","last_clean":"2024-09-13T09:10:18.185727+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:10:18.185727+0000","last_undegraded":"2024-09-13T09:10:18.185727+0000","last_fullsized":"2024-09-13T09:10:18.185727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017595029999999999,"stat_sum":{"num_bytes":827433,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":6779,"num_read_kb":25995,"num_write":797,"num_write_kb":6960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"16"}]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-13T09:10:22.218645+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:10:22.218645+0000","last_peered":"2024-09-13T09:10:22.218645+0000","last_clean":"2024-09-13T09:10:22.218645+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:10:22.218645+0000","last_undegraded":"2024-09-13T09:10:22.218645+0000","last_fullsized":"2024-09-13T09:10:22.218645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"184'701","reported_seq":3813,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.233984+0000","last_change":"2024-09-13T09:10:21.236440+0000","last_active":"2024-09-13T09:10:21.233984+0000","last_peered":"2024-09-13T09:10:21.233984+0000","last_clean":"2024-09-13T09:10:21.233984+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:10:21.233984+0000","last_undegraded":"2024-09-13T09:10:21.233984+0000","last_fullsized":"2024-09-13T09:10:21.233984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011296219,"stat_sum":{"num_bytes":590050,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":4905,"num_read_kb":19732,"num_write":886,"num_write_kb":10479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.3","version":"184'642","reported_seq":3439,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.421971+0000","last_change":"2024-09-13T09:10:21.237991+0000","last_active":"2024-09-13T09:10:21.421971+0000","last_peered":"2024-09-13T09:10:21.421971+0000","last_clean":"2024-09-13T09:10:21.421971+0000","last_became_active":"2024-09-13T09:10:19.196904+0000","last_became_peered":"2024-09-13T09:10:19.196904+0000","last_unstale":"2024-09-13T09:10:21.421971+0000","last_undegraded":"2024-09-13T09:10:21.421971+0000","last_fullsized":"2024-09-13T09:10:21.421971+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:07.676360+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015305786,"stat_sum":{"num_bytes":176128,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":3478,"num_read_kb":18381,"num_write":613,"num_write_kb":6198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"2.2","version":"21'2","reported_seq":182,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.201081+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:10:20.201081+0000","last_peered":"2024-09-13T09:10:20.201081+0000","last_clean":"2024-09-13T09:10:20.201081+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:10:20.201081+0000","last_undegraded":"2024-09-13T09:10:20.201081+0000","last_fullsized":"2024-09-13T09:10:20.201081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"186'497","reported_seq":3740,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.240947+0000","last_change":"2024-09-13T09:10:23.242625+0000","last_active":"2024-09-13T09:10:23.240947+0000","last_peered":"2024-09-13T09:10:23.240947+0000","last_clean":"2024-09-13T09:10:23.240947+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:10:23.240947+0000","last_undegraded":"2024-09-13T09:10:23.240947+0000","last_fullsized":"2024-09-13T09:10:23.240947+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:27.395379+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0085332110000000006,"stat_sum":{"num_bytes":3191247,"num_objects":26,"num_object_clones":1,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":20,"num_read":3582,"num_read_kb":19319,"num_write":500,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.284294+0000","last_change":"2024-09-13T09:10:19.191965+0000","last_active":"2024-09-13T09:10:20.284294+0000","last_peered":"2024-09-13T09:10:20.284294+0000","last_clean":"2024-09-13T09:10:20.284294+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:20.284294+0000","last_undegraded":"2024-09-13T09:10:20.284294+0000","last_fullsized":"2024-09-13T09:10:20.284294+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:17.687266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"177'414","reported_seq":2283,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-13T09:10:18.185233+0000","last_change":"2024-09-13T09:10:18.185233+0000","last_active":"2024-09-13T09:10:18.185233+0000","last_peered":"2024-09-13T09:10:18.185233+0000","last_clean":"2024-09-13T09:10:18.185233+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:10:18.185233+0000","last_undegraded":"2024-09-13T09:10:18.185233+0000","last_fullsized":"2024-09-13T09:10:18.185233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012894569999999999,"stat_sum":{"num_bytes":3291686,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":2069,"num_read_kb":14897,"num_write":408,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"16"}]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.200855+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:10:20.200855+0000","last_peered":"2024-09-13T09:10:20.200855+0000","last_clean":"2024-09-13T09:10:20.200855+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:10:20.200855+0000","last_undegraded":"2024-09-13T09:10:20.200855+0000","last_fullsized":"2024-09-13T09:10:20.200855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"177'391","reported_seq":2503,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.209398+0000","last_change":"2024-09-13T09:10:20.209924+0000","last_active":"2024-09-13T09:10:20.209398+0000","last_peered":"2024-09-13T09:10:20.209398+0000","last_clean":"2024-09-13T09:10:20.209398+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:10:20.209398+0000","last_undegraded":"2024-09-13T09:10:20.209398+0000","last_fullsized":"2024-09-13T09:10:20.209398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003769648,"stat_sum":{"num_bytes":1048632,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":2506,"num_read_kb":12208,"num_write":407,"num_write_kb":3728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"2.3","version":"19'1","reported_seq":181,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.202390+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:10:20.202390+0000","last_peered":"2024-09-13T09:10:20.202390+0000","last_clean":"2024-09-13T09:10:20.202390+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:10:20.202390+0000","last_undegraded":"2024-09-13T09:10:20.202390+0000","last_fullsized":"2024-09-13T09:10:20.202390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":310,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.195533+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:10:20.195533+0000","last_peered":"2024-09-13T09:10:20.195533+0000","last_clean":"2024-09-13T09:10:20.195533+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:10:20.195533+0000","last_undegraded":"2024-09-13T09:10:20.195533+0000","last_fullsized":"2024-09-13T09:10:20.195533+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"182'871","reported_seq":6085,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.205961+0000","last_change":"2024-09-13T09:10:20.207200+0000","last_active":"2024-09-13T09:10:20.205961+0000","last_peered":"2024-09-13T09:10:20.205961+0000","last_clean":"2024-09-13T09:10:20.205961+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:10:20.205961+0000","last_undegraded":"2024-09-13T09:10:20.205961+0000","last_fullsized":"2024-09-13T09:10:20.205961+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:33.786643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014141010000000001,"stat_sum":{"num_bytes":4939241,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":6851,"num_read_kb":23201,"num_write":1273,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.9","version":"183'567","reported_seq":3475,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.219761+0000","last_change":"2024-09-13T09:10:21.220650+0000","last_active":"2024-09-13T09:10:21.219761+0000","last_peered":"2024-09-13T09:10:21.219761+0000","last_clean":"2024-09-13T09:10:21.219761+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:21.219761+0000","last_undegraded":"2024-09-13T09:10:21.219761+0000","last_fullsized":"2024-09-13T09:10:21.219761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487904,"stat_sum":{"num_bytes":1474488,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":4630,"num_read_kb":16644,"num_write":668,"num_write_kb":9248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.a","version":"183'571","reported_seq":3036,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.216480+0000","last_change":"2024-09-13T09:10:20.216480+0000","last_active":"2024-09-13T09:10:20.216480+0000","last_peered":"2024-09-13T09:10:20.216480+0000","last_clean":"2024-09-13T09:10:20.216480+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:20.216480+0000","last_undegraded":"2024-09-13T09:10:20.216480+0000","last_fullsized":"2024-09-13T09:10:20.216480+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010462634,"stat_sum":{"num_bytes":4193559,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":3996,"num_read_kb":22625,"num_write":540,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.b","version":"183'418","reported_seq":3817,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.225436+0000","last_change":"2024-09-13T09:10:21.231678+0000","last_active":"2024-09-13T09:10:21.225436+0000","last_peered":"2024-09-13T09:10:21.225436+0000","last_clean":"2024-09-13T09:10:21.225436+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:10:21.225436+0000","last_undegraded":"2024-09-13T09:10:21.225436+0000","last_fullsized":"2024-09-13T09:10:21.225436+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:11.164172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069263160000000001,"stat_sum":{"num_bytes":6248852,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":13,"num_read":3594,"num_read_kb":33920,"num_write":419,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.c","version":"184'357","reported_seq":17017,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.224067+0000","last_change":"2024-09-13T09:10:21.224067+0000","last_active":"2024-09-13T09:10:21.224067+0000","last_peered":"2024-09-13T09:10:21.224067+0000","last_clean":"2024-09-13T09:10:21.224067+0000","last_became_active":"2024-09-13T09:10:19.195189+0000","last_became_peered":"2024-09-13T09:10:19.195189+0000","last_unstale":"2024-09-13T09:10:21.224067+0000","last_undegraded":"2024-09-13T09:10:21.224067+0000","last_fullsized":"2024-09-13T09:10:21.224067+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:34.127866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545654,"stat_sum":{"num_bytes":1960328,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":16596,"num_read_kb":27933,"num_write":360,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.d","version":"180'402","reported_seq":3078,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.237763+0000","last_change":"2024-09-13T09:10:21.238377+0000","last_active":"2024-09-13T09:10:21.237763+0000","last_peered":"2024-09-13T09:10:21.237763+0000","last_clean":"2024-09-13T09:10:21.237763+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:21.237763+0000","last_undegraded":"2024-09-13T09:10:21.237763+0000","last_fullsized":"2024-09-13T09:10:21.237763+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017967299999999999,"stat_sum":{"num_bytes":3946641,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":3095,"num_read_kb":9820,"num_write":356,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.e","version":"182'879","reported_seq":7081,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.236518+0000","last_change":"2024-09-13T09:10:21.238149+0000","last_active":"2024-09-13T09:10:21.236518+0000","last_peered":"2024-09-13T09:10:21.236518+0000","last_clean":"2024-09-13T09:10:21.236518+0000","last_became_active":"2024-09-13T09:08:38.205194+0000","last_became_peered":"2024-09-13T09:08:38.205194+0000","last_unstale":"2024-09-13T09:10:21.236518+0000","last_undegraded":"2024-09-13T09:10:21.236518+0000","last_fullsized":"2024-09-13T09:10:21.236518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:14.407799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010419016,"stat_sum":{"num_bytes":2874716,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":10,"num_read":9292,"num_read_kb":23776,"num_write":1220,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.f","version":"182'469","reported_seq":5110,"reported_epoch":184,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-13T09:10:21.229583+0000","last_change":"2024-09-13T09:10:21.229583+0000","last_active":"2024-09-13T09:10:21.229583+0000","last_peered":"2024-09-13T09:10:21.229583+0000","last_clean":"2024-09-13T09:10:21.229583+0000","last_became_active":"2024-09-13T09:09:53.883015+0000","last_became_peered":"2024-09-13T09:09:53.883015+0000","last_unstale":"2024-09-13T09:10:21.229583+0000","last_undegraded":"2024-09-13T09:10:21.229583+0000","last_fullsized":"2024-09-13T09:10:21.229583+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:38:48.295399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028292180000000001,"stat_sum":{"num_bytes":1544526,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":5016,"num_read_kb":17262,"num_write":423,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.10","version":"182'500","reported_seq":2830,"reported_epoch":184,"state":"active+clean+snaptrim","last_fresh":"2024-09-13T09:10:21.230958+0000","last_change":"2024-09-13T09:10:21.230991+0000","last_active":"2024-09-13T09:10:21.230958+0000","last_peered":"2024-09-13T09:10:21.230958+0000","last_clean":"2024-09-13T09:10:21.230958+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:10:21.230958+0000","last_undegraded":"2024-09-13T09:10:21.230958+0000","last_fullsized":"2024-09-13T09:10:21.230958+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012765400000000001,"stat_sum":{"num_bytes":6824006,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":18,"num_read":2878,"num_read_kb":17152,"num_write":480,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.11","version":"184'1161","reported_seq":4266,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.232345+0000","last_change":"2024-09-13T09:10:21.232939+0000","last_active":"2024-09-13T09:10:21.232345+0000","last_peered":"2024-09-13T09:10:21.232345+0000","last_clean":"2024-09-13T09:10:21.232345+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:21.232345+0000","last_undegraded":"2024-09-13T09:10:21.232345+0000","last_fullsized":"2024-09-13T09:10:21.232345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.014239579,"stat_sum":{"num_bytes":4253090,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":7659,"num_read_kb":12228,"num_write":1643,"num_write_kb":5381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.12","version":"182'691","reported_seq":4433,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-13T09:10:19.366945+0000","last_change":"2024-09-13T09:10:18.185942+0000","last_active":"2024-09-13T09:10:19.366945+0000","last_peered":"2024-09-13T09:10:19.366945+0000","last_clean":"2024-09-13T09:10:19.366945+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:19.366945+0000","last_undegraded":"2024-09-13T09:10:19.366945+0000","last_fullsized":"2024-09-13T09:10:19.366945+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001499261,"stat_sum":{"num_bytes":13640886,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":5588,"num_read_kb":13858,"num_write":951,"num_write_kb":11373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"16"}]},{"pgid":"3.13","version":"184'582","reported_seq":3818,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.234358+0000","last_change":"2024-09-13T09:10:21.235140+0000","last_active":"2024-09-13T09:10:21.234358+0000","last_peered":"2024-09-13T09:10:21.234358+0000","last_clean":"2024-09-13T09:10:21.234358+0000","last_became_active":"2024-09-13T09:09:04.005219+0000","last_became_peered":"2024-09-13T09:09:04.005219+0000","last_unstale":"2024-09-13T09:10:21.234358+0000","last_undegraded":"2024-09-13T09:10:21.234358+0000","last_fullsized":"2024-09-13T09:10:21.234358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:56.239198+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015419817000000001,"stat_sum":{"num_bytes":2752602,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":17,"num_read":3371,"num_read_kb":13500,"num_write":604,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":117178162,"num_objects":549,"num_object_clones":57,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":549,"num_whiteouts":388,"num_read":209224,"num_read_kb":689126,"num_write":24718,"num_write_kb":338025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2610,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68689920,"data_stored":84106155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":20961,"ondisk_log_size":20961,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080578,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":8876,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":9089024,"data_stored":7910105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":14756,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":15110144,"data_stored":19488611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64616904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.22,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.875,"5min":1.361,"15min":1.361},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63500000000000001,"15min":0.64700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.1459999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63600000000000001},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.88100000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":5.7080000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":1.444,"15min":1.444},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1850000000000001,"5min":1.766,"15min":1.766},"last":5.8109999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.8500000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2170000000000001,"5min":1.343,"15min":1.343},"last":2.5939999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.919,"15min":1.919},"last":5.8449999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.9119999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.90400000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4019999999999999,"5min":1.867,"15min":1.867},"last":5.6260000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.375}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.6319999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84696,"kb_used_data":17484,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94287144,"statfs":{"total":96636764160,"available":96550035456,"internally_reserved":0,"allocated":17903616,"data_stored":18356516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.67300000000000004,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.226,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73099999999999998,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.024,"5min":1.353,"15min":1.353},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.624},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.2350000000000001,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.94599999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.401,"5min":1.681,"15min":1.681},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.3,"5min":1.401,"15min":1.401},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":1.0249999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.288,"5min":1.998,"15min":1.998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.391,"5min":1.651,"15min":1.651},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4379999999999999,"5min":1.506,"15min":1.506},"last":1.077},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0009999999999999,"15min":1.0449999999999999},"min":{"1min":0.48899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.571,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.0580000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.071},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3500000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.167},{"interface":"front","average":{"1min":1.052,"5min":1.089,"15min":1.125},"min":{"1min":0.621,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.417,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":1.0009999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68600,"kb_used_data":15340,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94303240,"statfs":{"total":96636764160,"available":96566517760,"internally_reserved":0,"allocated":15708160,"data_stored":19043807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.30199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.86399999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.624,"15min":0.64400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.8859999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":2.7010000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.72399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0900000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.827},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3480000000000001,"5min":1.407,"15min":1.407},"last":0.81999999999999995}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3280000000000001,"5min":1.415,"15min":1.415},"last":3.7930000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.335,"5min":1.448,"15min":1.448},"last":1.9379999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94399999999999995,"15min":0.97899999999999998},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":2.306},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1539999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":3.351}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93100000000000005,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4510000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.05},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.49,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.0149999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":2.0880000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.8640000000000001}]}]},{"osd":0,"up_from":0,"seq":64424509485,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44960,"kb_used_data":9752,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94326880,"statfs":{"total":96636764160,"available":96590725120,"internally_reserved":0,"allocated":9986048,"data_stored":11532238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.434,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1759999999999999,"5min":1.552,"15min":1.552},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.446,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.79600000000000004},"min":{"1min":0.47799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2989999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.92000000000000004},"min":{"1min":0.53200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2669999999999999,"5min":2.706,"15min":2.706},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.93600000000000005,"15min":0.89700000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.649,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.497},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.58899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1439999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.92200000000000004,"15min":0.96399999999999997},"min":{"1min":0.498,"5min":0.377,"15min":0.377},"max":{"1min":1.5489999999999999,"5min":1.7,"15min":1.7},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.046,"15min":1.0800000000000001},"min":{"1min":0.5,"5min":0.497,"15min":0.497},"max":{"1min":1.359,"5min":1.776,"15min":1.776},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.04,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.6220000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.45100000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64252,"kb_used_data":6896,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307588,"statfs":{"total":96636764160,"available":96570970112,"internally_reserved":0,"allocated":7061504,"data_stored":5817497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.121,"5min":2.649,"15min":2.649},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.202,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1819999999999999,"5min":1.3,"15min":1.3},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.88,"15min":0.83099999999999996},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.042,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.91600000000000004,"15min":0.86799999999999999},"min":{"1min":0.623,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.591,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.505},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.98999999999999999,"15min":0.98199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.90100000000000002,"15min":0.85799999999999998},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.008,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":1.21,"5min":0.95899999999999996,"15min":0.89000000000000001},"min":{"1min":0.54900000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.74,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.98599999999999999,"15min":0.94699999999999995},"min":{"1min":0.44600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.042,"15min":1.0389999999999999},"min":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.58399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53044,"kb_used_data":7968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318796,"statfs":{"total":96636764160,"available":96582447104,"internally_reserved":0,"allocated":8159232,"data_stored":6945437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.032,"15min":1.1499999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1930000000000001,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.624}]},{"osd":1,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.83599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2490000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.335,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.78500000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":1.0269999999999999,"15min":1.1319999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2809999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.85599999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.1990000000000001},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3799999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.151,"5min":1.585,"15min":1.585},"last":1.002}]},{"osd":4,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.997,"15min":1.024},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4119999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.25,"15min":1.3799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.351,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.97999999999999998,"15min":0.98399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.278,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.355,"5min":1.899,"15min":1.899},"last":1.1930000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.272},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1419999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6295552,"data_stored":7615357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12816384,"data_stored":17292887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15532032,"data_stored":17115182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12795904,"data_stored":18333358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4722688,"data_stored":4630456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1896448,"data_stored":1839255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7753728,"data_stored":10437334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6877184,"data_stored":6842326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:10:24.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:10:24.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T09:10:24.739 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:24.737+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:24.740 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:24.740 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:24.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T09:10:24.838 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:24.839 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:24.839 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:24.833+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:24.869 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:24.870 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:24.887 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:24.887 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:24.904 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:24.904 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:24.932 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:24.932+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:24.936 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:24.936 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:24.939 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:24.939 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:24.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T09:10:24.964 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:24.965 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:24.984 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:24.984 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:24.989 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:10:25.004 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:10:25.004 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:10:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-13T09:10:24.245010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117768549,"num_objects":553,"num_object_clones":57,"num_object_copies":1106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553,"num_whiteouts":388,"num_read":209340,"num_read_kb":689226,"num_write":24879,"num_write_kb":340443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2612,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21081,"ondisk_log_size":21081,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":431184,"kb_used_data":81072,"kb_used_omap":29,"kb_used_meta":349986,"kb_avail":660171696,"statfs":{"total":676457349120,"available":676015816704,"internally_reserved":0,"allocated":83017728,"data_stored":89094211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29807,"internal_metadata":358386577},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-359692785,"num_objects":-177,"num_object_clones":-71,"num_object_copies":-354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-177,"num_whiteouts":-66,"num_read":1028,"num_read_kb":885,"num_write":167,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.056010"},"pg_stats":[{"pgid":"3.1f","version":"184'565","reported_seq":5686,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.241242+0000","last_change":"2024-09-13T09:10:23.241545+0000","last_active":"2024-09-13T09:10:23.241242+0000","last_peered":"2024-09-13T09:10:23.241242+0000","last_clean":"2024-09-13T09:10:23.241242+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:23.241242+0000","last_undegraded":"2024-09-13T09:10:23.241242+0000","last_fullsized":"2024-09-13T09:10:23.241242+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0073648230000000004,"stat_sum":{"num_bytes":2006600,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":5968,"num_read_kb":20217,"num_write":673,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"3.1e","version":"186'1202","reported_seq":4985,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.240432+0000","last_change":"2024-09-13T09:10:23.241104+0000","last_active":"2024-09-13T09:10:23.240432+0000","last_peered":"2024-09-13T09:10:23.240432+0000","last_clean":"2024-09-13T09:10:23.240432+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:10:23.240432+0000","last_undegraded":"2024-09-13T09:10:23.240432+0000","last_fullsized":"2024-09-13T09:10:23.240432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1202,"log_dups_size":0,"ondisk_log_size":1202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006939181,"stat_sum":{"num_bytes":262144,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":11,"num_read":9083,"num_read_kb":19020,"num_write":1553,"num_write_kb":6026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"3.1d","version":"184'1399","reported_seq":5363,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.239763+0000","last_change":"2024-09-13T09:10:21.240487+0000","last_active":"2024-09-13T09:10:21.239763+0000","last_peered":"2024-09-13T09:10:21.239763+0000","last_clean":"2024-09-13T09:10:21.239763+0000","last_became_active":"2024-09-13T09:08:38.208191+0000","last_became_peered":"2024-09-13T09:08:38.208191+0000","last_unstale":"2024-09-13T09:10:21.239763+0000","last_undegraded":"2024-09-13T09:10:21.239763+0000","last_fullsized":"2024-09-13T09:10:21.239763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1399,"log_dups_size":0,"ondisk_log_size":1399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:32:04.686300+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.02041157,"stat_sum":{"num_bytes":10466292,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":18,"num_read":8733,"num_read_kb":34149,"num_write":2230,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.1c","version":"183'722","reported_seq":4054,"reported_epoch":184,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-13T09:10:21.229270+0000","last_change":"2024-09-13T09:10:21.229253+0000","last_active":"2024-09-13T09:10:21.229270+0000","last_peered":"2024-09-13T09:10:21.229270+0000","last_clean":"2024-09-13T09:10:21.229270+0000","last_became_active":"2024-09-13T09:08:38.209451+0000","last_became_peered":"2024-09-13T09:08:38.209451+0000","last_unstale":"2024-09-13T09:10:21.229270+0000","last_undegraded":"2024-09-13T09:10:21.229270+0000","last_fullsized":"2024-09-13T09:10:21.229270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:59.672566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022080989999999998,"stat_sum":{"num_bytes":1048576,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":5703,"num_read_kb":35589,"num_write":957,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.1b","version":"183'710","reported_seq":22123,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.343459+0000","last_change":"2024-09-13T09:10:20.208684+0000","last_active":"2024-09-13T09:10:20.343459+0000","last_peered":"2024-09-13T09:10:20.343459+0000","last_clean":"2024-09-13T09:10:20.343459+0000","last_became_active":"2024-09-13T09:08:38.210333+0000","last_became_peered":"2024-09-13T09:08:38.210333+0000","last_unstale":"2024-09-13T09:10:20.343459+0000","last_undegraded":"2024-09-13T09:10:20.343459+0000","last_fullsized":"2024-09-13T09:10:20.343459+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:14:53.493070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026289769999999998,"stat_sum":{"num_bytes":2299681,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":23645,"num_read_kb":48156,"num_write":772,"num_write_kb":18569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.1a","version":"183'698","reported_seq":6432,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.230930+0000","last_change":"2024-09-13T09:10:21.230930+0000","last_active":"2024-09-13T09:10:21.230930+0000","last_peered":"2024-09-13T09:10:21.230930+0000","last_clean":"2024-09-13T09:10:21.230930+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:10:21.230930+0000","last_undegraded":"2024-09-13T09:10:21.230930+0000","last_fullsized":"2024-09-13T09:10:21.230930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017851049999999999,"stat_sum":{"num_bytes":2004071,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":15,"num_read":8408,"num_read_kb":25820,"num_write":846,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.19","version":"178'384","reported_seq":2057,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.206126+0000","last_change":"2024-09-13T09:10:20.207203+0000","last_active":"2024-09-13T09:10:20.206126+0000","last_peered":"2024-09-13T09:10:20.206126+0000","last_clean":"2024-09-13T09:10:20.206126+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:10:20.206126+0000","last_undegraded":"2024-09-13T09:10:20.206126+0000","last_fullsized":"2024-09-13T09:10:20.206126+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012737,"stat_sum":{"num_bytes":450789,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":15,"num_read":1827,"num_read_kb":8140,"num_write":347,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.18","version":"184'711","reported_seq":3531,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.241687+0000","last_change":"2024-09-13T09:10:23.243272+0000","last_active":"2024-09-13T09:10:23.241687+0000","last_peered":"2024-09-13T09:10:23.241687+0000","last_clean":"2024-09-13T09:10:23.241687+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:10:23.241687+0000","last_undegraded":"2024-09-13T09:10:23.241687+0000","last_fullsized":"2024-09-13T09:10:23.241687+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0088804250000000008,"stat_sum":{"num_bytes":1042388,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":4805,"num_read_kb":14792,"num_write":762,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"3.17","version":"184'644","reported_seq":5828,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.242688+0000","last_change":"2024-09-13T09:10:23.243259+0000","last_active":"2024-09-13T09:10:23.242688+0000","last_peered":"2024-09-13T09:10:23.242688+0000","last_clean":"2024-09-13T09:10:23.242688+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:10:23.242688+0000","last_undegraded":"2024-09-13T09:10:23.242688+0000","last_fullsized":"2024-09-13T09:10:23.242688+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0087551869999999993,"stat_sum":{"num_bytes":675672,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":7182,"num_read_kb":31245,"num_write":619,"num_write_kb":18265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"3.16","version":"184'700","reported_seq":2055,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.241499+0000","last_change":"2024-09-13T09:10:21.242266+0000","last_active":"2024-09-13T09:10:21.241499+0000","last_peered":"2024-09-13T09:10:21.241499+0000","last_clean":"2024-09-13T09:10:21.241499+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:21.241499+0000","last_undegraded":"2024-09-13T09:10:21.241499+0000","last_fullsized":"2024-09-13T09:10:21.241499+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.016871398999999999,"stat_sum":{"num_bytes":4537583,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":10,"num_read":2905,"num_read_kb":14193,"num_write":951,"num_write_kb":9806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.15","version":"178'695","reported_seq":4229,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-13T09:10:18.186196+0000","last_change":"2024-09-13T09:10:18.186196+0000","last_active":"2024-09-13T09:10:18.186196+0000","last_peered":"2024-09-13T09:10:18.186196+0000","last_clean":"2024-09-13T09:10:18.186196+0000","last_became_active":"2024-09-13T09:08:38.221484+0000","last_became_peered":"2024-09-13T09:08:38.221484+0000","last_unstale":"2024-09-13T09:10:18.186196+0000","last_undegraded":"2024-09-13T09:10:18.186196+0000","last_fullsized":"2024-09-13T09:10:18.186196+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:27:04.676858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013267699999999999,"stat_sum":{"num_bytes":13390952,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":5093,"num_read_kb":24615,"num_write":673,"num_write_kb":16161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"16"}]},{"pgid":"3.14","version":"184'698","reported_seq":19349,"reported_epoch":184,"state":"active+clean+snaptrim","last_fresh":"2024-09-13T09:10:21.241925+0000","last_change":"2024-09-13T09:10:21.230427+0000","last_active":"2024-09-13T09:10:21.241925+0000","last_peered":"2024-09-13T09:10:21.241925+0000","last_clean":"2024-09-13T09:10:21.241925+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:10:21.241925+0000","last_undegraded":"2024-09-13T09:10:21.241925+0000","last_fullsized":"2024-09-13T09:10:21.241925+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.001800718,"stat_sum":{"num_bytes":12416133,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":21094,"num_read_kb":34154,"num_write":614,"num_write_kb":10193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.216771+0000","last_change":"2024-09-13T09:08:35.188587+0000","last_active":"2024-09-13T09:10:20.216771+0000","last_peered":"2024-09-13T09:10:20.216771+0000","last_clean":"2024-09-13T09:10:20.216771+0000","last_became_active":"2024-09-13T09:08:35.188229+0000","last_became_peered":"2024-09-13T09:08:35.188229+0000","last_unstale":"2024-09-13T09:10:20.216771+0000","last_undegraded":"2024-09-13T09:10:20.216771+0000","last_fullsized":"2024-09-13T09:10:20.216771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:37:13.481119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"183'690","reported_seq":5937,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.209148+0000","last_change":"2024-09-13T09:10:20.209148+0000","last_active":"2024-09-13T09:10:20.209148+0000","last_peered":"2024-09-13T09:10:20.209148+0000","last_clean":"2024-09-13T09:10:20.209148+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:10:20.209148+0000","last_undegraded":"2024-09-13T09:10:20.209148+0000","last_fullsized":"2024-09-13T09:10:20.209148+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:13.066567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030828689999999998,"stat_sum":{"num_bytes":1048576,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":6169,"num_read_kb":21942,"num_write":788,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"2.6","version":"0'0","reported_seq":187,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-13T09:10:19.195120+0000","last_change":"2024-09-13T09:08:35.184636+0000","last_active":"2024-09-13T09:10:19.195120+0000","last_peered":"2024-09-13T09:10:19.195120+0000","last_clean":"2024-09-13T09:10:19.195120+0000","last_became_active":"2024-09-13T09:08:35.183626+0000","last_became_peered":"2024-09-13T09:08:35.183626+0000","last_unstale":"2024-09-13T09:10:19.195120+0000","last_undegraded":"2024-09-13T09:10:19.195120+0000","last_fullsized":"2024-09-13T09:10:19.195120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:03:39.924062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"182'424","reported_seq":2475,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.208303+0000","last_change":"2024-09-13T09:10:20.208303+0000","last_active":"2024-09-13T09:10:20.208303+0000","last_peered":"2024-09-13T09:10:20.208303+0000","last_clean":"2024-09-13T09:10:20.208303+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:10:20.208303+0000","last_undegraded":"2024-09-13T09:10:20.208303+0000","last_fullsized":"2024-09-13T09:10:20.208303+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023577979999999999,"stat_sum":{"num_bytes":1750594,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":17,"num_read":2724,"num_read_kb":14643,"num_write":385,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"2.5","version":"0'0","reported_seq":180,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.201671+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:10:20.201671+0000","last_peered":"2024-09-13T09:10:20.201671+0000","last_clean":"2024-09-13T09:10:20.201671+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:10:20.201671+0000","last_undegraded":"2024-09-13T09:10:20.201671+0000","last_fullsized":"2024-09-13T09:10:20.201671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"180'606","reported_seq":6165,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-13T09:10:18.185727+0000","last_change":"2024-09-13T09:10:18.185727+0000","last_active":"2024-09-13T09:10:18.185727+0000","last_peered":"2024-09-13T09:10:18.185727+0000","last_clean":"2024-09-13T09:10:18.185727+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:10:18.185727+0000","last_undegraded":"2024-09-13T09:10:18.185727+0000","last_fullsized":"2024-09-13T09:10:18.185727+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017595029999999999,"stat_sum":{"num_bytes":827433,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":9,"num_read":6779,"num_read_kb":25995,"num_write":797,"num_write_kb":6960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"16"}]},{"pgid":"2.4","version":"0'0","reported_seq":182,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-13T09:10:22.218645+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:10:22.218645+0000","last_peered":"2024-09-13T09:10:22.218645+0000","last_clean":"2024-09-13T09:10:22.218645+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:10:22.218645+0000","last_undegraded":"2024-09-13T09:10:22.218645+0000","last_fullsized":"2024-09-13T09:10:22.218645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"184'701","reported_seq":3813,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.233984+0000","last_change":"2024-09-13T09:10:21.236440+0000","last_active":"2024-09-13T09:10:21.233984+0000","last_peered":"2024-09-13T09:10:21.233984+0000","last_clean":"2024-09-13T09:10:21.233984+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:10:21.233984+0000","last_undegraded":"2024-09-13T09:10:21.233984+0000","last_fullsized":"2024-09-13T09:10:21.233984+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.011296219,"stat_sum":{"num_bytes":590050,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":4905,"num_read_kb":19732,"num_write":886,"num_write_kb":10479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.3","version":"184'642","reported_seq":3439,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.421971+0000","last_change":"2024-09-13T09:10:21.237991+0000","last_active":"2024-09-13T09:10:21.421971+0000","last_peered":"2024-09-13T09:10:21.421971+0000","last_clean":"2024-09-13T09:10:21.421971+0000","last_became_active":"2024-09-13T09:10:19.196904+0000","last_became_peered":"2024-09-13T09:10:19.196904+0000","last_unstale":"2024-09-13T09:10:21.421971+0000","last_undegraded":"2024-09-13T09:10:21.421971+0000","last_fullsized":"2024-09-13T09:10:21.421971+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:07.676360+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.015305786,"stat_sum":{"num_bytes":176128,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":3478,"num_read_kb":18381,"num_write":613,"num_write_kb":6198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"2.2","version":"21'2","reported_seq":182,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.201081+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:10:20.201081+0000","last_peered":"2024-09-13T09:10:20.201081+0000","last_clean":"2024-09-13T09:10:20.201081+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:10:20.201081+0000","last_undegraded":"2024-09-13T09:10:20.201081+0000","last_fullsized":"2024-09-13T09:10:20.201081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"186'497","reported_seq":3740,"reported_epoch":186,"state":"active+clean","last_fresh":"2024-09-13T09:10:23.240947+0000","last_change":"2024-09-13T09:10:23.242625+0000","last_active":"2024-09-13T09:10:23.240947+0000","last_peered":"2024-09-13T09:10:23.240947+0000","last_clean":"2024-09-13T09:10:23.240947+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:10:23.240947+0000","last_undegraded":"2024-09-13T09:10:23.240947+0000","last_fullsized":"2024-09-13T09:10:23.240947+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:27.395379+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0085332110000000006,"stat_sum":{"num_bytes":3191247,"num_objects":26,"num_object_clones":1,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":20,"num_read":3582,"num_read_kb":19319,"num_write":500,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"1a"}]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.284294+0000","last_change":"2024-09-13T09:10:19.191965+0000","last_active":"2024-09-13T09:10:20.284294+0000","last_peered":"2024-09-13T09:10:20.284294+0000","last_clean":"2024-09-13T09:10:20.284294+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:20.284294+0000","last_undegraded":"2024-09-13T09:10:20.284294+0000","last_fullsized":"2024-09-13T09:10:20.284294+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:17.687266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"177'414","reported_seq":2283,"reported_epoch":181,"state":"active+clean","last_fresh":"2024-09-13T09:10:18.185233+0000","last_change":"2024-09-13T09:10:18.185233+0000","last_active":"2024-09-13T09:10:18.185233+0000","last_peered":"2024-09-13T09:10:18.185233+0000","last_clean":"2024-09-13T09:10:18.185233+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:10:18.185233+0000","last_undegraded":"2024-09-13T09:10:18.185233+0000","last_fullsized":"2024-09-13T09:10:18.185233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012894569999999999,"stat_sum":{"num_bytes":3291686,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":2069,"num_read_kb":14897,"num_write":408,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"16"}]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.200855+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:10:20.200855+0000","last_peered":"2024-09-13T09:10:20.200855+0000","last_clean":"2024-09-13T09:10:20.200855+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:10:20.200855+0000","last_undegraded":"2024-09-13T09:10:20.200855+0000","last_fullsized":"2024-09-13T09:10:20.200855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"177'391","reported_seq":2503,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.209398+0000","last_change":"2024-09-13T09:10:20.209924+0000","last_active":"2024-09-13T09:10:20.209398+0000","last_peered":"2024-09-13T09:10:20.209398+0000","last_clean":"2024-09-13T09:10:20.209398+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:10:20.209398+0000","last_undegraded":"2024-09-13T09:10:20.209398+0000","last_fullsized":"2024-09-13T09:10:20.209398+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003769648,"stat_sum":{"num_bytes":1048632,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":2506,"num_read_kb":12208,"num_write":407,"num_write_kb":3728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"2.3","version":"19'1","reported_seq":181,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.202390+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:10:20.202390+0000","last_peered":"2024-09-13T09:10:20.202390+0000","last_clean":"2024-09-13T09:10:20.202390+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:10:20.202390+0000","last_undegraded":"2024-09-13T09:10:20.202390+0000","last_fullsized":"2024-09-13T09:10:20.202390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":310,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.195533+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:10:20.195533+0000","last_peered":"2024-09-13T09:10:20.195533+0000","last_clean":"2024-09-13T09:10:20.195533+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:10:20.195533+0000","last_undegraded":"2024-09-13T09:10:20.195533+0000","last_fullsized":"2024-09-13T09:10:20.195533+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"182'871","reported_seq":6085,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.205961+0000","last_change":"2024-09-13T09:10:20.207200+0000","last_active":"2024-09-13T09:10:20.205961+0000","last_peered":"2024-09-13T09:10:20.205961+0000","last_clean":"2024-09-13T09:10:20.205961+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:10:20.205961+0000","last_undegraded":"2024-09-13T09:10:20.205961+0000","last_fullsized":"2024-09-13T09:10:20.205961+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:33.786643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014141010000000001,"stat_sum":{"num_bytes":4939241,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":15,"num_read":6851,"num_read_kb":23201,"num_write":1273,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.9","version":"183'567","reported_seq":3475,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.219761+0000","last_change":"2024-09-13T09:10:21.220650+0000","last_active":"2024-09-13T09:10:21.219761+0000","last_peered":"2024-09-13T09:10:21.219761+0000","last_clean":"2024-09-13T09:10:21.219761+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:21.219761+0000","last_undegraded":"2024-09-13T09:10:21.219761+0000","last_fullsized":"2024-09-13T09:10:21.219761+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001487904,"stat_sum":{"num_bytes":1474488,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":4630,"num_read_kb":16644,"num_write":668,"num_write_kb":9248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.a","version":"183'571","reported_seq":3036,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-09-13T09:10:20.216480+0000","last_change":"2024-09-13T09:10:20.216480+0000","last_active":"2024-09-13T09:10:20.216480+0000","last_peered":"2024-09-13T09:10:20.216480+0000","last_clean":"2024-09-13T09:10:20.216480+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:20.216480+0000","last_undegraded":"2024-09-13T09:10:20.216480+0000","last_fullsized":"2024-09-13T09:10:20.216480+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010462634,"stat_sum":{"num_bytes":4193559,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":3996,"num_read_kb":22625,"num_write":540,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.b","version":"183'418","reported_seq":3817,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.225436+0000","last_change":"2024-09-13T09:10:21.231678+0000","last_active":"2024-09-13T09:10:21.225436+0000","last_peered":"2024-09-13T09:10:21.225436+0000","last_clean":"2024-09-13T09:10:21.225436+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:10:21.225436+0000","last_undegraded":"2024-09-13T09:10:21.225436+0000","last_fullsized":"2024-09-13T09:10:21.225436+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:11.164172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0069263160000000001,"stat_sum":{"num_bytes":6248852,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":13,"num_read":3594,"num_read_kb":33920,"num_write":419,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.c","version":"184'357","reported_seq":17017,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.224067+0000","last_change":"2024-09-13T09:10:21.224067+0000","last_active":"2024-09-13T09:10:21.224067+0000","last_peered":"2024-09-13T09:10:21.224067+0000","last_clean":"2024-09-13T09:10:21.224067+0000","last_became_active":"2024-09-13T09:10:19.195189+0000","last_became_peered":"2024-09-13T09:10:19.195189+0000","last_unstale":"2024-09-13T09:10:21.224067+0000","last_undegraded":"2024-09-13T09:10:21.224067+0000","last_fullsized":"2024-09-13T09:10:21.224067+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:34.127866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001545654,"stat_sum":{"num_bytes":1960328,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":13,"num_read":16596,"num_read_kb":27933,"num_write":360,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.d","version":"180'402","reported_seq":3078,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.237763+0000","last_change":"2024-09-13T09:10:21.238377+0000","last_active":"2024-09-13T09:10:21.237763+0000","last_peered":"2024-09-13T09:10:21.237763+0000","last_clean":"2024-09-13T09:10:21.237763+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:21.237763+0000","last_undegraded":"2024-09-13T09:10:21.237763+0000","last_fullsized":"2024-09-13T09:10:21.237763+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017967299999999999,"stat_sum":{"num_bytes":3946641,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":3095,"num_read_kb":9820,"num_write":356,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.e","version":"182'879","reported_seq":7081,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.236518+0000","last_change":"2024-09-13T09:10:21.238149+0000","last_active":"2024-09-13T09:10:21.236518+0000","last_peered":"2024-09-13T09:10:21.236518+0000","last_clean":"2024-09-13T09:10:21.236518+0000","last_became_active":"2024-09-13T09:08:38.205194+0000","last_became_peered":"2024-09-13T09:08:38.205194+0000","last_unstale":"2024-09-13T09:10:21.236518+0000","last_undegraded":"2024-09-13T09:10:21.236518+0000","last_fullsized":"2024-09-13T09:10:21.236518+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:53:14.407799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010419016,"stat_sum":{"num_bytes":2874716,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":10,"num_read":9292,"num_read_kb":23776,"num_write":1220,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.f","version":"182'469","reported_seq":5110,"reported_epoch":184,"state":"active+clean+snaptrim_wait","last_fresh":"2024-09-13T09:10:21.229583+0000","last_change":"2024-09-13T09:10:21.229583+0000","last_active":"2024-09-13T09:10:21.229583+0000","last_peered":"2024-09-13T09:10:21.229583+0000","last_clean":"2024-09-13T09:10:21.229583+0000","last_became_active":"2024-09-13T09:09:53.883015+0000","last_became_peered":"2024-09-13T09:09:53.883015+0000","last_unstale":"2024-09-13T09:10:21.229583+0000","last_undegraded":"2024-09-13T09:10:21.229583+0000","last_fullsized":"2024-09-13T09:10:21.229583+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:38:48.295399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028292180000000001,"stat_sum":{"num_bytes":1544526,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":5016,"num_read_kb":17262,"num_write":423,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.10","version":"182'500","reported_seq":2830,"reported_epoch":184,"state":"active+clean+snaptrim","last_fresh":"2024-09-13T09:10:21.230958+0000","last_change":"2024-09-13T09:10:21.230991+0000","last_active":"2024-09-13T09:10:21.230958+0000","last_peered":"2024-09-13T09:10:21.230958+0000","last_clean":"2024-09-13T09:10:21.230958+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:10:21.230958+0000","last_undegraded":"2024-09-13T09:10:21.230958+0000","last_fullsized":"2024-09-13T09:10:21.230958+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012765400000000001,"stat_sum":{"num_bytes":6824006,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":18,"num_read":2878,"num_read_kb":17152,"num_write":480,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"b","length":"17"}]},{"pgid":"3.11","version":"184'1161","reported_seq":4266,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.232345+0000","last_change":"2024-09-13T09:10:21.232939+0000","last_active":"2024-09-13T09:10:21.232345+0000","last_peered":"2024-09-13T09:10:21.232345+0000","last_clean":"2024-09-13T09:10:21.232345+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:21.232345+0000","last_undegraded":"2024-09-13T09:10:21.232345+0000","last_fullsized":"2024-09-13T09:10:21.232345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.014239579,"stat_sum":{"num_bytes":4253090,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":9,"num_read":7659,"num_read_kb":12228,"num_write":1643,"num_write_kb":5381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]},{"pgid":"3.12","version":"182'691","reported_seq":4433,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-09-13T09:10:19.366945+0000","last_change":"2024-09-13T09:10:18.185942+0000","last_active":"2024-09-13T09:10:19.366945+0000","last_peered":"2024-09-13T09:10:19.366945+0000","last_clean":"2024-09-13T09:10:19.366945+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:19.366945+0000","last_undegraded":"2024-09-13T09:10:19.366945+0000","last_fullsized":"2024-09-13T09:10:19.366945+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":691,"log_dups_size":0,"ondisk_log_size":691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001499261,"stat_sum":{"num_bytes":13640886,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":5588,"num_read_kb":13858,"num_write":951,"num_write_kb":11373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"c","length":"16"}]},{"pgid":"3.13","version":"184'582","reported_seq":3818,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-09-13T09:10:21.234358+0000","last_change":"2024-09-13T09:10:21.235140+0000","last_active":"2024-09-13T09:10:21.234358+0000","last_peered":"2024-09-13T09:10:21.234358+0000","last_clean":"2024-09-13T09:10:21.234358+0000","last_became_active":"2024-09-13T09:09:04.005219+0000","last_became_peered":"2024-09-13T09:09:04.005219+0000","last_unstale":"2024-09-13T09:10:21.234358+0000","last_undegraded":"2024-09-13T09:10:21.234358+0000","last_fullsized":"2024-09-13T09:10:21.234358+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:55:56.239198+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.015419817000000001,"stat_sum":{"num_bytes":2752602,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":17,"num_read":3371,"num_read_kb":13500,"num_write":604,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"18"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":117178162,"num_objects":549,"num_object_clones":57,"num_object_copies":1098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":549,"num_whiteouts":388,"num_read":209224,"num_read_kb":689126,"num_write":24718,"num_write_kb":338025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2610,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68689920,"data_stored":84106155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":20961,"ondisk_log_size":20961,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080578,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37756,"kb_used_data":8876,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94334084,"statfs":{"total":96636764160,"available":96598102016,"internally_reserved":0,"allocated":9089024,"data_stored":7910105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77876,"kb_used_data":14756,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94293964,"statfs":{"total":96636764160,"available":96557019136,"internally_reserved":0,"allocated":15110144,"data_stored":19488611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64616904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.22,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.875,"5min":1.361,"15min":1.361},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63500000000000001,"15min":0.64700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.1459999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63600000000000001},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.88100000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":5.7080000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":1.444,"15min":1.444},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1850000000000001,"5min":1.766,"15min":1.766},"last":5.8109999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.8500000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2170000000000001,"5min":1.343,"15min":1.343},"last":2.5939999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.919,"15min":1.919},"last":5.8449999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.9119999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.90400000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4019999999999999,"5min":1.867,"15min":1.867},"last":5.6260000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.375}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.6319999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84696,"kb_used_data":17484,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94287144,"statfs":{"total":96636764160,"available":96550035456,"internally_reserved":0,"allocated":17903616,"data_stored":18356516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.67300000000000004,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.226,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73099999999999998,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.024,"5min":1.353,"15min":1.353},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.624},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.2350000000000001,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.94599999999999995}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.401,"5min":1.681,"15min":1.681},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.3,"5min":1.401,"15min":1.401},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":1.0249999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.288,"5min":1.998,"15min":1.998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.391,"5min":1.651,"15min":1.651},"last":1.1020000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4379999999999999,"5min":1.506,"15min":1.506},"last":1.077},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0009999999999999,"15min":1.0449999999999999},"min":{"1min":0.48899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.571,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.0580000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.071},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3500000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.167},{"interface":"front","average":{"1min":1.052,"5min":1.089,"15min":1.125},"min":{"1min":0.621,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.417,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.2110000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":1.0009999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68600,"kb_used_data":15340,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94303240,"statfs":{"total":96636764160,"available":96566517760,"internally_reserved":0,"allocated":15708160,"data_stored":19043807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.30199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.86399999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":1.3049999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.624,"15min":0.64400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.8859999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":2.7010000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.72399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0900000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.827},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3480000000000001,"5min":1.407,"15min":1.407},"last":0.81999999999999995}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3280000000000001,"5min":1.415,"15min":1.415},"last":3.7930000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.335,"5min":1.448,"15min":1.448},"last":1.9379999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94399999999999995,"15min":0.97899999999999998},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":2.306},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1539999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":3.351}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93100000000000005,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4510000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.05},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.49,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.0149999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":2.0880000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.8640000000000001}]}]},{"osd":0,"up_from":0,"seq":64424509485,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44960,"kb_used_data":9752,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94326880,"statfs":{"total":96636764160,"available":96590725120,"internally_reserved":0,"allocated":9986048,"data_stored":11532238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.434,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1759999999999999,"5min":1.552,"15min":1.552},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.446,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.79600000000000004},"min":{"1min":0.47799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2989999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.92000000000000004},"min":{"1min":0.53200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2669999999999999,"5min":2.706,"15min":2.706},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.93600000000000005,"15min":0.89700000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.649,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.497},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.58899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1439999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.92200000000000004,"15min":0.96399999999999997},"min":{"1min":0.498,"5min":0.377,"15min":0.377},"max":{"1min":1.5489999999999999,"5min":1.7,"15min":1.7},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.046,"15min":1.0800000000000001},"min":{"1min":0.5,"5min":0.497,"15min":0.497},"max":{"1min":1.359,"5min":1.776,"15min":1.776},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.04,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.6220000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.45100000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64252,"kb_used_data":6896,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94307588,"statfs":{"total":96636764160,"available":96570970112,"internally_reserved":0,"allocated":7061504,"data_stored":5817497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.121,"5min":2.649,"15min":2.649},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.202,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1819999999999999,"5min":1.3,"15min":1.3},"last":0.60799999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.88,"15min":0.83099999999999996},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.042,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.65400000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.91600000000000004,"15min":0.86799999999999999},"min":{"1min":0.623,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.591,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.505},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.98999999999999999,"15min":0.98199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.90100000000000002,"15min":0.85799999999999998},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.008,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":1.21,"5min":0.95899999999999996,"15min":0.89000000000000001},"min":{"1min":0.54900000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.74,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.98599999999999999,"15min":0.94699999999999995},"min":{"1min":0.44600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.042,"15min":1.0389999999999999},"min":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":0.58399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53044,"kb_used_data":7968,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318796,"statfs":{"total":96636764160,"available":96582447104,"internally_reserved":0,"allocated":8159232,"data_stored":6945437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.032,"15min":1.1499999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1930000000000001,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.624}]},{"osd":1,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.83599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2490000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.877,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.335,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.78500000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":1.0269999999999999,"15min":1.1319999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2809999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.85599999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.1990000000000001},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3799999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.151,"5min":1.585,"15min":1.585},"last":1.002}]},{"osd":4,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.997,"15min":1.024},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4119999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.25,"15min":1.3799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.351,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.97999999999999998,"15min":0.98399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.278,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.355,"5min":1.899,"15min":1.899},"last":1.1930000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.272},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.1419999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6295552,"data_stored":7615357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12816384,"data_stored":17292887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15532032,"data_stored":17115182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12795904,"data_stored":18333358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4722688,"data_stored":4630456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1896448,"data_stored":1839255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7753728,"data_stored":10437334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6877184,"data_stored":6842326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:10:25.016 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:10:25.016 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T09:10:25.016 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-13T09:10:25.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-13T09:10:25.064 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:25.061+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55588 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2989056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.978746879, 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056694643000000003, 2024-09-13T09:10:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55588 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112535+0000", 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112537+0000", 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.322e-06 2024-09-13T09:10:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112544+0000", 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4710000000000001e-06 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112551+0000", 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9299999999999997e-06 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169079+0000", 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056527276000000001 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169155+0000", 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6574000000000002e-05 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169230+0000", 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4388000000000007e-05 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55589 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3003392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.978726112, 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056853167000000003, 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55589 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112556+0000", 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112555+0000", 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112558+0000", 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112564+0000", 2024-09-13T09:10:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0639999999999999e-06 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112569+0000", 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169255+0000", 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056686328000000001 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169337+0000", 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169409+0000", 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55590 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3027968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97870825999999, 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057010392, 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55590 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112574+0000", 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112573+0000", 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112575+0000", 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112581+0000", 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0310000000000004e-06 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112586+0000", 2024-09-13T09:10:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3970000000000004e-06 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169437+0000", 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056851226999999997 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169508+0000", 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0958999999999996e-05 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169584+0000", 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5964999999999995e-05 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55591 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3042304~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97869169399999, 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057168073999999999, 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55591 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112590+0000", 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112592+0000", 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5059999999999998e-06 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112598+0000", 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.178e-06 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112603+0000", 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8470000000000003e-06 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169614+0000", 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057011339000000001 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169688+0000", 2024-09-13T09:10:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3114999999999994e-05 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169758+0000", 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0650999999999998e-05 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55592 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3056640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97866610899999, 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057320358000000002, 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55592 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112616+0000", 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112615+0000", 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112618+0000", 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9100000000000001e-06 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112625+0000", 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112630+0000", 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.356e-06 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169786+0000", 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057155607999999997 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169858+0000", 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2436999999999998e-05 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169936+0000", 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7799999999999994e-05 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55593 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3070976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97864694200001, 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057480834000000001, 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55593 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112635+0000", 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112634+0000", 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112637+0000", 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000002e-06 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112643+0000", 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0759999999999999e-06 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112648+0000", 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7779999999999998e-06 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.169965+0000", 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057317273000000002 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170039+0000", 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4005999999999996e-05 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170116+0000", 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6518000000000002e-05 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55594 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3085312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97862965900001, 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057648209999999998, 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55594 2024-09-13T09:10:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112652+0000", 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112654+0000", 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.198e-06 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112666+0000", 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1714e-05 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112672+0000", 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5989999999999998e-06 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170145+0000", 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057472857000000002 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170222+0000", 2024-09-13T09:10:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7113000000000005e-05 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170300+0000", 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8077999999999993e-05 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55595 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3099648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.978100789, 2024-09-13T09:10:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057296802000000001, 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55595 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113181+0000", 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.112677+0000", 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9994955 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113186+0000", 2024-09-13T09:10:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050942299999999995 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113198+0000", 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1963999999999999e-05 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113206+0000", 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5190000000000003e-06 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170329+0000", 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057123452999999998 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170402+0000", 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2880999999999996e-05 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170478+0000", 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5931000000000002e-05 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55596 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3113984~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97806888400001, 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057439530000000003, 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55596 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113213+0000", 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113212+0000", 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113220+0000", 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7000000000000008e-06 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113228+0000", 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6590000000000007e-06 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113233+0000", 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7540000000000003e-06 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170507+0000", 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057273615 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170578+0000", 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1258000000000001e-05 2024-09-13T09:10:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170652+0000", 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4177999999999994e-05 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55597 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3128320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97804393600001, 2024-09-13T09:10:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057588827000000002, 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55597 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113238+0000", 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113241+0000", 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1690000000000001e-06 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113247+0000", 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3620000000000002e-06 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113252+0000", 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170680+0000", 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057427907 2024-09-13T09:10:25.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170751+0000", 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0959999999999998e-05 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170827+0000", 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5568000000000006e-05 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55598 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3142656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97773937100001, 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057468566999999998, 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55598 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113542+0000", 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113257+0000", 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997144 2024-09-13T09:10:25.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113547+0000", 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029064500000000002 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113559+0000", 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1585e-05 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.113567+0000", 2024-09-13T09:10:25.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1960000000000008e-06 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170856+0000", 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057288956000000002 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.170931+0000", 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4611999999999996e-05 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171011+0000", 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0348999999999999e-05 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55611 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2947072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:25.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97597694800001, 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056616090000000001, 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55611 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115305+0000", 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115304+0000", 2024-09-13T09:10:25.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115308+0000", 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3249999999999999e-06 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115315+0000", 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6820000000000003e-06 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115321+0000", 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.609e-06 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171775+0000", 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056453929999999999 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171847+0000", 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2466999999999996e-05 2024-09-13T09:10:25.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171921+0000", 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3713000000000004e-05 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55612 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2961408~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97595609800001, 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056771060999999998, 2024-09-13T09:10:25.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55612 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115326+0000", 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115325+0000", 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115328+0000", 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6139999999999999e-06 2024-09-13T09:10:25.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115338+0000", 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9189999999999994e-06 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115343+0000", 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5289999999999999e-06 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.171950+0000", 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056607007000000001 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172028+0000", 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7202999999999999e-05 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172097+0000", 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9122999999999995e-05 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55613 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.975933774, 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056933663000000002, 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55613 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115348+0000", 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115350+0000", 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.627e-06 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115357+0000", 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0820000000000001e-06 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115362+0000", 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.921e-06 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172123+0000", 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056760509000000001 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172200+0000", 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7248000000000003e-05 2024-09-13T09:10:25.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172282+0000", 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1620999999999997e-05 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55614 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2990080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.975915085, 2024-09-13T09:10:25.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057089110999999998, 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55614 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115367+0000", 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115366+0000", 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115369+0000", 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3779999999999999e-06 2024-09-13T09:10:25.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115375+0000", 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9030000000000001e-06 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115380+0000", 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9169999999999997e-06 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172311+0000", 2024-09-13T09:10:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.056930910000000001 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172387+0000", 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6593000000000003e-05 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172456+0000", 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8737000000000003e-05 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55615 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3004416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97589803299999, 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057262293999999998, 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55615 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115384+0000", 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115383+0000", 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115386+0000", 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115391+0000", 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6540000000000004e-06 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115396+0000", 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7140000000000001e-06 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172487+0000", 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057090632000000002 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172564+0000", 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7656000000000003e-05 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172646+0000", 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1777e-05 2024-09-13T09:10:25.164 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55616 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 3014656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.97588100900001, 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057425347000000002, 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.165 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55616 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115401+0000", 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115400+0000", 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115403+0000", 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.362e-06 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115409+0000", 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1460000000000001e-06 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.115414+0000", 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0529999999999997e-06 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172676+0000", 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.057261812000000002 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172753+0000", 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7021000000000007e-05 2024-09-13T09:10:25.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.172826+0000", 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3313999999999996e-05 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:55699 3.c 3:327ed144:::rbd_data.1124c595d92c.0000000000000002:14 [read 2991104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 144.965868022, 2024-09-13T09:10:25.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060444391, 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 55699 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125414+0000", 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125413+0000", 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125416+0000", 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.368e-06 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125422+0000", 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2559999999999999e-06 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.125428+0000", 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3319999999999997e-06 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179502+0000", 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054074022999999999 2024-09-13T09:10:25.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.179577+0000", 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4264000000000006e-05 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.185858+0000", 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062815019999999996 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122577 3.f 3.80993c2f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 103.58571349899999, 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74211478399999997, 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122577 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505568+0000", 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505566+0000", 2024-09-13T09:10:25.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505578+0000", 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2038999999999999e-05 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505606+0000", 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8299000000000001e-05 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505619+0000", 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2503e-05 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505704+0000", 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4808000000000005e-05 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505716+0000", 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1878e-05 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247649+0000", 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74193310800000001 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247683+0000", 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4353000000000001e-05 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:122583 3.f 3.79bb412f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:25.178 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 103.58553307699999, 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74208591300000004, 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 122583 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505749+0000", 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505747+0000", 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505757+0000", 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0366999999999999e-05 2024-09-13T09:10:25.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505778+0000", 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0463999999999999e-05 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505789+0000", 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1528e-05 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505838+0000", 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8383000000000001e-05 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:41.505847+0000", 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9020000000000005e-06 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247805+0000", 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.74195847699999995 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:42.247835+0000", 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9615999999999999e-05 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.183 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.183 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:25.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T09:10:25.185 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-09-13T09:10:25.181+0000 7efe50a91640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26196) UID: 0 2024-09-13T09:10:25.284 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:25.281+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:25.351 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:25.351 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:25.351 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:25.351 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:25.364 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T09:10:25.376 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-09-13T09:10:25.373+0000 7f426cbbd640 -1 mon.a@0(leader).osd e188 definitely_dead 0 2024-09-13T09:10:25.385 INFO:tasks.ceph.osd.5.smithi060.stderr:2024-09-13T09:10:25.384+0000 7f3f5de41640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26340) UID: 0 2024-09-13T09:10:25.486 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:25.481+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:25.491 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:25.492 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:25.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T09:10:25.587 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:25.584+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154208 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.559616848999994, 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019831428, 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.674 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154208 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110900+0000", 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110898+0000", 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110909+0000", 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0896999999999999e-05 2024-09-13T09:10:25.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110932+0000", 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3544999999999999e-05 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110943+0000", 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.064e-05 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119027+0000", 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080840670000000003 2024-09-13T09:10:25.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119033+0000", 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0050000000000001e-06 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130694+0000", 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0116608949999999 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130732+0000", 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7696e-05 2024-09-13T09:10:25.678 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154209 3.14 3.1dd2fbb4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.559558792000004, 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199906329999999, 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.679 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154209 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:25.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110958+0000", 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110961+0000", 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0450000000000001e-06 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110967+0000", 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4470000000000003e-06 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.110972+0000", 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119044+0000", 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080713150000000008 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119046+0000", 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6910000000000002e-06 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130938+0000", 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0118917460000001 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130949+0000", 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0607e-05 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154218 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.559482911999993, 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019944518, 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154218 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111034+0000", 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111033+0000", 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111038+0000", 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2350000000000001e-06 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111047+0000", 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1779999999999992e-06 2024-09-13T09:10:25.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111053+0000", 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9089999999999997e-06 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119060+0000", 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080075760000000006 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119065+0000", 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.578e-06 2024-09-13T09:10:25.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130971+0000", 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119057490000001 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130979+0000", 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8790000000000003e-06 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154219 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.559441132000003, 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0199227849999999, 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154219 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111076+0000", 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111075+0000", 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111079+0000", 2024-09-13T09:10:25.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2830000000000002e-06 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111086+0000", 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8900000000000001e-06 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111090+0000", 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7859999999999996e-06 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119079+0000", 2024-09-13T09:10:25.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.007988396 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119081+0000", 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5569999999999998e-06 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130995+0000", 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011913643 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.130999+0000", 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6940000000000001e-06 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154228 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.559100775999994, 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.019595799, 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154228 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111416+0000", 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111415+0000", 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111421+0000", 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1460000000000004e-06 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111432+0000", 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1807000000000001e-05 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.111439+0000", 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8909999999999998e-06 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119092+0000", 2024-09-13T09:10:25.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076528159999999998 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119097+0000", 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.87e-06 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131008+0000", 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011911539 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131012+0000", 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5760000000000002e-06 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154229 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558335161000002, 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01884818, 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154229 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112182+0000", 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112181+0000", 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112186+0000", 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249999999999997e-06 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112202+0000", 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.543e-05 2024-09-13T09:10:25.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112209+0000", 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5669999999999997e-06 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119115+0000", 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069053999999999999 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119120+0000", 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.215e-06 2024-09-13T09:10:25.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131023+0000", 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011903446 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131030+0000", 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6100000000000002e-06 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154238 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558300849999995, 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188343550000001, 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154238 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:25.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112216+0000", 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112219+0000", 2024-09-13T09:10:25.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0240000000000002e-06 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112225+0000", 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3980000000000004e-06 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112230+0000", 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0159999999999999e-06 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119130+0000", 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068996450000000003 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119132+0000", 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6400000000000001e-06 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131046+0000", 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119135260000001 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131050+0000", 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5859999999999998e-06 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154239 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558282375999994, 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188390510000001, 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154239 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112235+0000", 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112234+0000", 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112237+0000", 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4279999999999999e-06 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112243+0000", 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8510000000000004e-06 2024-09-13T09:10:25.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112247+0000", 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1980000000000003e-06 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119149+0000", 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069020679999999999 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119154+0000", 2024-09-13T09:10:25.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1989999999999997e-06 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131067+0000", 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011912659 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131074+0000", 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9709999999999998e-06 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154242 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558262974000002, 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188413620000001, 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154242 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112254+0000", 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112256+0000", 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112262+0000", 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9200000000000001e-06 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112267+0000", 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4240000000000003e-06 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119166+0000", 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068991599999999997 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119168+0000", 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.678e-06 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131088+0000", 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011919435 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131095+0000", 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5769999999999996e-06 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154243 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558235121999999, 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018836657, 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154243 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:25.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112282+0000", 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112281+0000", 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112285+0000", 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8480000000000003e-06 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112292+0000", 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.126e-06 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112297+0000", 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0760000000000002e-06 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119178+0000", 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068802890000000004 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119180+0000", 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5100000000000001e-06 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131111+0000", 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011930947 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131119+0000", 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154244 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.558214973999995, 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0188321789999999, 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154244 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112302+0000", 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112304+0000", 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3240000000000001e-06 2024-09-13T09:10:25.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112310+0000", 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1859999999999997e-06 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112315+0000", 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6070000000000001e-06 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119189+0000", 2024-09-13T09:10:25.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068746950000000001 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119192+0000", 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131128+0000", 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01193613 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131134+0000", 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0240000000000003e-06 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154251 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.557616584000002, 2024-09-13T09:10:25.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018254725, 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154251 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112900+0000", 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112899+0000", 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112905+0000", 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5709999999999997e-06 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112916+0000", 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0798999999999999e-05 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.112923+0000", 2024-09-13T09:10:25.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2339999999999997e-06 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119201+0000", 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062780379999999997 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119204+0000", 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5600000000000001e-06 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131151+0000", 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011947269 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131155+0000", 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.284e-06 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.731 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154252 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.556475160000005, 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171325529999999, 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154252 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114042+0000", 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114041+0000", 2024-09-13T09:10:25.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114046+0000", 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1610000000000002e-06 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114062+0000", 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5951999999999999e-05 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114070+0000", 2024-09-13T09:10:25.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9510000000000006e-06 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119213+0000", 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051428860000000002 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119215+0000", 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4509999999999999e-06 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131171+0000", 2024-09-13T09:10:25.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119552249999999 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131174+0000", 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6739999999999999e-06 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154253 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:25.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.556439949999998, 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017116229, 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154253 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114077+0000", 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114080+0000", 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114087+0000", 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1679999999999998e-06 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114092+0000", 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119225+0000", 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051325920000000001 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119227+0000", 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5189999999999999e-06 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131187+0000", 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011959702 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131193+0000", 2024-09-13T09:10:25.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3999999999999997e-06 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154260 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.556403191000001, 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017101802, 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154260 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114114+0000", 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114113+0000", 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114117+0000", 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3139999999999999e-06 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114125+0000", 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1429999999999998e-06 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114131+0000", 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119237+0000", 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051056859999999999 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119239+0000", 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131212+0000", 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119728100000001 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131216+0000", 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7160000000000001e-06 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154271 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.556380735999994, 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017099328, 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154271 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114136+0000", 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114138+0000", 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5390000000000002e-06 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114145+0000", 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1129999999999997e-06 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114150+0000", 2024-09-13T09:10:25.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9579999999999998e-06 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119251+0000", 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051011809999999998 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119256+0000", 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.989e-06 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131232+0000", 2024-09-13T09:10:25.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011976298 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131236+0000", 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6030000000000001e-06 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154272 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:25.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.556363056999999, 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.017104572, 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154272 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114154+0000", 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114156+0000", 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3939999999999999e-06 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114162+0000", 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0700000000000003e-06 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114167+0000", 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.977e-06 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119274+0000", 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051065360000000001 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119279+0000", 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0989999999999998e-06 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131252+0000", 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011972874 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131259+0000", 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154273 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.556345213, 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171116979999999, 2024-09-13T09:10:25.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154273 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114172+0000", 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114171+0000", 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114174+0000", 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5720000000000001e-06 2024-09-13T09:10:25.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114181+0000", 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5030000000000002e-06 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114185+0000", 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6720000000000003e-06 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119297+0000", 2024-09-13T09:10:25.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051116549999999997 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119302+0000", 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5570000000000003e-06 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131276+0000", 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.011973451 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131283+0000", 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.627e-06 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154274 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.556328913000002, 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171203449999999, 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154274 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114188+0000", 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967296 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114190+0000", 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7519999999999999e-06 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114193+0000", 2024-09-13T09:10:25.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3400000000000002e-06 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.114195+0000", 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.407e-06 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119320+0000", 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051245739999999998 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119323+0000", 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.14e-06 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131301+0000", 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.01197782 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131308+0000", 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5440000000000001e-06 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:154293 3.14 3.1dd2fbb4 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e122)", 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 66.553300609999994, 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0141231429999999, 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:25.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 154293 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117216+0000", 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117215+0000", 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117223+0000", 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2169999999999994e-06 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117236+0000", 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3492000000000001e-05 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.117247+0000", 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0725e-05 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119334+0000", 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002087392 2024-09-13T09:10:25.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:19.119337+0000", 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5909999999999998e-06 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131332+0000", 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119952649999999 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:20.131340+0000", 2024-09-13T09:10:25.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2089999999999999e-06 2024-09-13T09:10:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:25.770 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:25.770 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:25.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T09:10:25.772 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:25.769+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:25.872 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:25.868+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:25.952 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:25.952 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:25.952 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:25.952 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:25.966 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T09:10:26.012 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:26.009+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:26.074 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:26.069+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:26.092 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:26.092 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:26.092 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:26.092 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:26.092 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:26.106 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T09:10:26.175 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:26.172+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:26.275 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20617 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5406720~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.934870434, 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043203130999999999, 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.276 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20617 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338584+0000", 2024-09-13T09:10:26.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338583+0000", 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338591+0000", 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4220000000000003e-06 2024-09-13T09:10:26.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338600+0000", 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6689999999999995e-06 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338615+0000", 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5389000000000001e-05 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381579+0000", 2024-09-13T09:10:26.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.042963897000000001 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381672+0000", 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2862000000000006e-05 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381787+0000", 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011547700000000001 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20618 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5439488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.934833129, 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043355080999999997, 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20618 2024-09-13T09:10:26.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338621+0000", 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338624+0000", 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5920000000000001e-06 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338632+0000", 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8029999999999997e-06 2024-09-13T09:10:26.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338637+0000", 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8380000000000001e-06 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381832+0000", 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043195457 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381909+0000", 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6221000000000001e-05 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.381976+0000", 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7840000000000001e-05 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20619 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5488640~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93481341500001, 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043586468000000003, 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20619 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338641+0000", 2024-09-13T09:10:26.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338643+0000", 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4669999999999998e-06 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338649+0000", 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338654+0000", 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5530000000000003e-06 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382024+0000", 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043370644 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382112+0000", 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7944999999999999e-05 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382228+0000", 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115085 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20620 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5521408~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93479708999999, 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043752081999999998, 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20620 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338657+0000", 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338659+0000", 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3029999999999998e-06 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338665+0000", 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5500000000000002e-06 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338669+0000", 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5879999999999999e-06 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382268+0000", 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043598971 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382322+0000", 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4005999999999998e-05 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382409+0000", 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7012e-05 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20621 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5603328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:26.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93478122400001, 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043934783999999998, 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20621 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338673+0000", 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338675+0000", 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.086e-06 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338681+0000", 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5899999999999998e-06 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338685+0000", 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3370000000000001e-06 2024-09-13T09:10:26.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382441+0000", 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043756309 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382516+0000", 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5246000000000004e-05 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382608+0000", 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1574000000000001e-05 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20622 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5636096~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93476543899999, 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044161873999999997, 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20622 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338689+0000", 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338691+0000", 2024-09-13T09:10:26.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1519999999999999e-06 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338696+0000", 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6629999999999998e-06 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338701+0000", 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6210000000000003e-06 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382647+0000", 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.043946188999999997 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382735+0000", 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7460999999999995e-05 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382851+0000", 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116185 2024-09-13T09:10:26.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20623 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5685248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93474636799999, 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044397535000000002, 2024-09-13T09:10:26.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20623 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338708+0000", 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338710+0000", 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2369999999999999e-06 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338716+0000", 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0179999999999998e-06 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338720+0000", 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-09-13T09:10:26.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382905+0000", 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044184052000000001 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.382997+0000", 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2404999999999994e-05 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383106+0000", 2024-09-13T09:10:26.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108707 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20624 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5718016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93472982099999, 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044660336000000002, 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.309 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338725+0000", 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338724+0000", 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338735+0000", 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1077000000000001e-05 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338743+0000", 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4039999999999998e-06 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338747+0000", 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6020000000000002e-06 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383159+0000", 2024-09-13T09:10:26.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044411249999999999 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383252+0000", 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3422000000000001e-05 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383385+0000", 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013297 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20625 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5799936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.934702222, 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044870833999999998, 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20625 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338752+0000", 2024-09-13T09:10:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338754+0000", 2024-09-13T09:10:26.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6989999999999999e-06 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338761+0000", 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0100000000000001e-06 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338765+0000", 2024-09-13T09:10:26.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6129999999999997e-06 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383429+0000", 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044664029000000001 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383516+0000", 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6408000000000003e-05 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383623+0000", 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107518 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20626 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5832704~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93468497999999, 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045086563000000003, 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20626 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338769+0000", 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338772+0000", 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5679999999999998e-06 2024-09-13T09:10:26.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338779+0000", 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.712e-06 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338794+0000", 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4955e-05 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383663+0000", 2024-09-13T09:10:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.044868997000000001 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383749+0000", 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5600999999999996e-05 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383856+0000", 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000107123 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20627 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5881856~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.934654537, 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045308867000000003, 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20627 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338800+0000", 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338799+0000", 2024-09-13T09:10:26.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338803+0000", 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0639999999999998e-06 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338810+0000", 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1879999999999996e-06 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338815+0000", 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9980000000000003e-06 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383906+0000", 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045090802999999999 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.383983+0000", 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7624000000000001e-05 2024-09-13T09:10:26.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384109+0000", 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012566300000000001 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20628 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5914624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.934635622, 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045509247000000003, 2024-09-13T09:10:26.328 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20628 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338819+0000", 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338818+0000", 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338821+0000", 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3800000000000001e-06 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338828+0000", 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6329999999999999e-06 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338832+0000", 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8369999999999996e-06 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384142+0000", 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045309338999999997 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384224+0000", 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1934000000000005e-05 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384328+0000", 2024-09-13T09:10:26.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104468 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20629 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 5996544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93461783399999, 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045712392999999997, 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20629 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338837+0000", 2024-09-13T09:10:26.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338836+0000", 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338845+0000", 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338852+0000", 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0140000000000001e-06 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338857+0000", 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3369999999999999e-06 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384359+0000", 2024-09-13T09:10:26.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045501682000000002 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384448+0000", 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9525999999999996e-05 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384549+0000", 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010053800000000001 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20630 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6029312~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93459261000001, 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045858138999999999, 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20630 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338862+0000", 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338861+0000", 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338864+0000", 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3520000000000001e-06 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338870+0000", 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.365e-06 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338875+0000", 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9080000000000003e-06 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384594+0000", 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045719017000000001 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384641+0000", 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7321999999999999e-05 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384720+0000", 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8583000000000003e-05 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20631 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6078464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.934574406, 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046084412999999998, 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20631 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338880+0000", 2024-09-13T09:10:26.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338882+0000", 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5830000000000001e-06 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338890+0000", 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338895+0000", 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7629999999999997e-06 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384765+0000", 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.045869524000000002 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384853+0000", 2024-09-13T09:10:26.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8523999999999995e-05 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.384964+0000", 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111113 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20632 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6111232~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:26.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.934554088, 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287383000000001, 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20632 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:26.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338900+0000", 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338902+0000", 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6560000000000001e-06 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338909+0000", 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9689999999999997e-06 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338915+0000", 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999997e-06 2024-09-13T09:10:26.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385011+0000", 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046096658999999998 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385100+0000", 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9151000000000001e-05 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385188+0000", 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7330000000000006e-05 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20633 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6193152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93453554000001, 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046399534999999999, 2024-09-13T09:10:26.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20633 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338919+0000", 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338921+0000", 2024-09-13T09:10:26.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2749999999999998e-06 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338927+0000", 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4740000000000002e-06 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338932+0000", 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.758e-06 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385219+0000", 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046287414999999998 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385263+0000", 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385318+0000", 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5133000000000002e-05 2024-09-13T09:10:26.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20634 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6225920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.934518266, 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046608632999999997, 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20634 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338936+0000", 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338938+0000", 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2840000000000001e-06 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338945+0000", 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4219999999999997e-06 2024-09-13T09:10:26.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338949+0000", 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385356+0000", 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046407055000000003 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385440+0000", 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3226999999999994e-05 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385545+0000", 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010511299999999999 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:20635 3.e 3:7586cf06:::rbd_data.1124d0fab35.0000000000000006:c [read 6275072~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:26.361 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 170.93450081200001, 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.04674793, 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20635 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338954+0000", 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338953+0000", 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338955+0000", 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1229999999999998e-06 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338962+0000", 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2009999999999996e-06 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.338976+0000", 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4076999999999999e-05 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385588+0000", 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046612268999999998 2024-09-13T09:10:26.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385636+0000", 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8362999999999998e-05 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:35.385702+0000", 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.525e-05 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:162508 3.b 3.9f9e59eb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e141)", 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.865959048000001, 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.47181327899999997, 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 162508 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407495+0000", 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407494+0000", 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407503+0000", 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0699999999999996e-06 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407523+0000", 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9692999999999999e-05 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407535+0000", 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2689e-05 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407598+0000", 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2064999999999996e-05 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.407606+0000", 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9560000000000003e-06 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.879280+0000", 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.47167379599999998 2024-09-13T09:10:26.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:38.879309+0000", 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8436999999999999e-05 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.370 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:26.370 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:26.273+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:26.371 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.0. 2024-09-13T09:10:26.372 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T09:10:26.373 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-13T09:10:26.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T09:10:26.377 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:26.376+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:26.477 INFO:tasks.ceph.osd.6.smithi060.stderr:2024-09-13T09:10:26.476+0000 7fb1d20a6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26339) UID: 0 2024-09-13T09:10:26.552 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:26.552 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-09-13T09:10:26.552 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:26.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T09:10:26.619 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-09-13T09:10:26.617+0000 7f0f4302b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26198) UID: 0 2024-09-13T09:10:26.679 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-09-13T09:10:26.673+0000 7fab77684640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26194) UID: 0 2024-09-13T09:10:26.686 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-09-13T09:10:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-09-13T09:10:26.687 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-09-13T09:10:26.687 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:26.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T09:10:26.780 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:26.776+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:26.867 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-09-13T09:10:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-09-13T09:10:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-09-13T09:10:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4235.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-13T09:10:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:10:26.867 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 202.732331894, 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.22363803700000001, 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4235", 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/4258411157", 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130731+0000", 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130729+0000", 2024-09-13T09:10:26.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130742+0000", 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2652e-05 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130773+0000", 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1056999999999998e-05 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130795+0000", 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2512000000000001e-05 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130882+0000", 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6986999999999999e-05 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.130895+0000", 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.205e-05 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351415+0000", 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.220520472 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.351696+0000", 2024-09-13T09:10:26.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028142800000000002 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.352032+0000", 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033520400000000002 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.353477+0000", 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001445626 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354197+0000", 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071988399999999995 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354298+0000", 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010040299999999999 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:04.354369+0000", 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1965999999999995e-05 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.874 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:39078 3.a 3:514f5ff3:::rbd_data.1124d4e7fcc5.000000000000004f:head [sparse-read 0~2097152] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 153.904049714, 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90586481600000002, 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 39078 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959014+0000", 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959013+0000", 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959018+0000", 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999999e-06 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959027+0000", 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959032+0000", 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0119999999999996e-06 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959077+0000", 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5053999999999999e-05 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:52.959083+0000", 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864709+0000", 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.90562572100000005 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:07:53.864878+0000", 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169588 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57029 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4363264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.880 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.61160987599999, 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092590747000000001, 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57029 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251453+0000", 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251462+0000", 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5890000000000003e-06 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251468+0000", 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5490000000000003e-06 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251473+0000", 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9710000000000003e-06 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343837+0000", 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092363974000000001 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.343912+0000", 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5211999999999997e-05 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344044+0000", 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013176400000000001 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57030 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5142528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.61153007799999, 2024-09-13T09:10:26.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092725021000000005, 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57030 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251533+0000", 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251477+0000", 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999442 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251536+0000", 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8866000000000002e-05 2024-09-13T09:10:26.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251545+0000", 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5229999999999995e-06 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251551+0000", 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344071+0000", 2024-09-13T09:10:26.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092520056000000003 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344146+0000", 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4691999999999996e-05 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344258+0000", 2024-09-13T09:10:26.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000112409 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57031 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4377600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.611281007, 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092690788999999996, 2024-09-13T09:10:26.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57031 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251782+0000", 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251556+0000", 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999774 2024-09-13T09:10:26.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251787+0000", 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023094399999999999 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251799+0000", 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1501000000000001e-05 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251806+0000", 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1779999999999997e-06 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344291+0000", 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092485523 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344357+0000", 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6112999999999995e-05 2024-09-13T09:10:26.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344473+0000", 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115677 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57032 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4391936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.61096786600001, 2024-09-13T09:10:26.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092605357999999999, 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57032 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252095+0000", 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.251812+0000", 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997163 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252100+0000", 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028827300000000002 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252109+0000", 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2269999999999999e-06 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252116+0000", 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2169999999999999e-06 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344501+0000", 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092385659999999994 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344577+0000", 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5430000000000001e-05 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344701+0000", 2024-09-13T09:10:26.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012411900000000001 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57033 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5156864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.610514634, 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092377856999999994, 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.900 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57033 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252549+0000", 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252121+0000", 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9995728 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252553+0000", 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000431302 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252561+0000", 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5390000000000007e-06 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252568+0000", 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2659999999999998e-06 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344728+0000", 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092160652999999995 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344802+0000", 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3359e-05 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344927+0000", 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124974 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57034 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4406272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.61049012000001, 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092586522000000004, 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57034 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252573+0000", 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252575+0000", 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.481e-06 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252581+0000", 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2480000000000004e-06 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252585+0000", 2024-09-13T09:10:26.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.143e-06 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.344954+0000", 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092369681999999995 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345025+0000", 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0182e-05 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345160+0000", 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135155 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57035 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 5171200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.610224891, 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092532444000000005, 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57035 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252838+0000", 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252588+0000", 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997501 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252843+0000", 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025450400000000001 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252852+0000", 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9979999999999992e-06 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252859+0000", 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5930000000000002e-06 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345188+0000", 2024-09-13T09:10:26.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092329261999999995 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345262+0000", 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4343000000000003e-05 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345371+0000", 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010877100000000001 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57036 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4420608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.60993704200001, 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092456405000000005, 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.914 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57036 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253126+0000", 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.252864+0000", 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9997377 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253131+0000", 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026665 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253140+0000", 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9609999999999994e-06 2024-09-13T09:10:26.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253146+0000", 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.686e-06 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345398+0000", 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092251386000000005 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345467+0000", 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9413999999999996e-05 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345583+0000", 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000115418 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:57038 3.14 3:2862b1e4:::rbd_data.1124c595d92c.0000000000000001:14 [read 4434944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-13T09:10:26.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 146.60947573799999, 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092429979999999995, 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 57038 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:10:26.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253588+0000", 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253587+0000", 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999995 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253590+0000", 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4339999999999999e-06 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253596+0000", 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.253601+0000", 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7690000000000004e-06 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345831+0000", 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.092230575999999995 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.345900+0000", 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8338999999999997e-05 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:00.346018+0000", 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011801700000000001 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119420 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e85)", 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.670942389, 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.018585173, 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119420 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192121+0000", 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192120+0000", 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192127+0000", 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7759999999999997e-06 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192140+0000", 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3021e-05 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192150+0000", 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.004e-05 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192251+0000", 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100778 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.192456+0000", 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000204825 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.193598+0000", 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001141483 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.195603+0000", 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020052249999999998 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202632+0000", 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0070296009999999999 2024-09-13T09:10:26.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.202650+0000", 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7323999999999999e-05 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210420+0000", 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0077706879999999 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.210706+0000", 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028573599999999998 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119421 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.65994906900001, 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0171641709999999, 2024-09-13T09:10:26.931 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119421 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203114+0000", 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203112+0000", 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203126+0000", 2024-09-13T09:10:26.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3743999999999999e-05 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203154+0000", 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.817e-05 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203170+0000", 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203244+0000", 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3651000000000003e-05 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.203257+0000", 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3529e-05 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211088+0000", 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0078310079999999 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.211268+0000", 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017956 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.213657+0000", 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023894630000000001 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217083+0000", 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00342604 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220130+0000", 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030462419999999998 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220244+0000", 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011474800000000001 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220278+0000", 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4122999999999998e-05 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119422 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.655967171, 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0143997140000001, 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119422 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207096+0000", 2024-09-13T09:10:26.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207094+0000", 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999976 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207107+0000", 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2819e-05 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207134+0000", 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7514000000000001e-05 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207152+0000", 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7452000000000002e-05 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207221+0000", 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9031999999999999e-05 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.207234+0000", 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2892e-05 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215519+0000", 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00828542 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.215661+0000", 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014215300000000001 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.216187+0000", 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00052627000000000001 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221359+0000", 2024-09-13T09:10:26.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051720940000000003 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221372+0000", 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2924e-05 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221441+0000", 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9010999999999994e-05 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221496+0000", 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4466999999999999e-05 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119423 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.65333917700001, 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0119571780000001, 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119423 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209724+0000", 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209722+0000", 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999981 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209732+0000", 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6579999999999997e-06 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209752+0000", 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9457999999999999e-05 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209762+0000", 2024-09-13T09:10:26.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0182e-05 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209833+0000", 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1604000000000001e-05 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.209843+0000", 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8800000000000003e-06 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217143+0000", 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0072996489999999 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217267+0000", 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012366699999999999 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:26.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.217547+0000", 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00028001000000000002 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221508+0000", 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039613670000000004 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221574+0000", 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6358000000000004e-05 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221626+0000", 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2172000000000001e-05 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221681+0000", 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4911000000000003e-05 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.952 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119424 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.65178639299999, 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0147389179999999, 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119424 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211277+0000", 2024-09-13T09:10:26.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211276+0000", 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.999999 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211282+0000", 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4500000000000001e-06 2024-09-13T09:10:26.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211293+0000", 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0224000000000001e-05 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211299+0000", 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211358+0000", 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9840000000000003e-05 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.211366+0000", 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8839999999999994e-06 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218270+0000", 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0069040330000001 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.218410+0000", 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139686 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.219021+0000", 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061123999999999998 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221515+0000", 2024-09-13T09:10:26.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002493267 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225797+0000", 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042828509999999998 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225935+0000", 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013764699999999999 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226016+0000", 2024-09-13T09:10:26.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0856e-05 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119425 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.65053307700001, 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136507189999999, 2024-09-13T09:10:26.960 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119425 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212530+0000", 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212529+0000", 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212539+0000", 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0616e-05 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212562+0000", 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2728e-05 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212572+0000", 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0376e-05 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212639+0000", 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6458999999999995e-05 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.212649+0000", 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9639999999999998e-06 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220322+0000", 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00767264 2024-09-13T09:10:26.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220386+0000", 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4865999999999999e-05 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.220636+0000", 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024998199999999999 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222504+0000", 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018675339999999999 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226084+0000", 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035803829999999999 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226139+0000", 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4922e-05 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226181+0000", 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1743000000000003e-05 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119426 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.649431044, 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0136407860000001, 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119426 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213632+0000", 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213631+0000", 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213642+0000", 2024-09-13T09:10:26.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1731e-05 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213674+0000", 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1303999999999997e-05 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213688+0000", 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4745e-05 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213762+0000", 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3514000000000001e-05 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.213769+0000", 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4950000000000002e-06 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221717+0000", 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.00794717 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.221823+0000", 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106221 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:26.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222087+0000", 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000263985 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.225708+0000", 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036208379999999999 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227171+0000", 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001463256 2024-09-13T09:10:26.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227223+0000", 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244000000000003e-05 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.227273+0000", 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9892000000000003e-05 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:119427 3.1d 3:bafbec3f:::rbd_object_map.11241fc9ca17:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 109.648220467, 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0151358189999999, 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 119427 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214843+0000", 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214842+0000", 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999986 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214851+0000", 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1719999999999996e-06 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214868+0000", 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7186000000000001e-05 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214877+0000", 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8289999999999997e-06 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214940+0000", 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3279000000000002e-05 2024-09-13T09:10:26.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:37.214948+0000", 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8369999999999997e-06 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222585+0000", 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.007637036 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.222865+0000", 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027978600000000002 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.224057+0000", 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00119224 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.226207+0000", 2024-09-13T09:10:26.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021502840000000001 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229874+0000", 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036663300000000002 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229934+0000", 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0581000000000002e-05 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:08:38.229979+0000", 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4549000000000003e-05 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4388.0:149046 3.f 3:f1cafc87:::rbd_data.1124651f5f97.000000000000004a:head [set-alloc-hint object_size 1048576 write_size 1048576,write 557056~22319] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 71.740525051999995, 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97276468599999999, 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T09:10:26.982 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4388", 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/1693557493", 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 149046 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-09-13T09:10:26.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122538+0000", 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122535+0000", 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4294967295.9999971 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122565+0000", 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9362000000000001e-05 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-09-13T09:10:26.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122590+0000", 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5596999999999999e-05 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122602+0000", 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1972e-05 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122685+0000", 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2354000000000005e-05 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:15.122701+0000", 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6107e-05 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092738+0000", 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.97003675 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.092877+0000", 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139245 2024-09-13T09:10:26.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093029+0000", 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015224999999999999 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.093847+0000", 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081788800000000001 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095250+0000", 2024-09-13T09:10:26.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014028529999999999 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095282+0000", 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1961000000000001e-05 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-09-13T09:09:16.095303+0000", 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1257999999999999e-05 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.988 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.989 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-13T09:10:26.989 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-13T09:10:26.989 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:26.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T09:10:26.991 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:26.988+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:27.022 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T09:10:27.091 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-09-13T09:10:27.089+0000 7f7375756640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26192) UID: 0 2024-09-13T09:10:27.125 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-13T09:10:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-13T09:10:27.125 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-13T09:10:27.125 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-13T09:10:27.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T09:10:27.180 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-09-13T09:10:27.180 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T09:10:27.191 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:27.188+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:27.200 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-09-13T09:10:27.200 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T09:10:27.218 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-09-13T09:10:27.218 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T09:10:27.235 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-09-13T09:10:27.235 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T09:10:27.255 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-09-13T09:10:27.256 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T09:10:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:27.270 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:27.276 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-09-13T09:10:27.276 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T09:10:27.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T09:10:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-09-13T09:10:27.304 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-09-13T09:10:27.304 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T09:10:27.329 INFO:tasks.ceph.osd.4.smithi060.stderr:2024-09-13T09:10:27.328+0000 7f6a64d24640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26337) UID: 0 2024-09-13T09:10:27.329 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-09-13T09:10:27.329 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T09:10:27.393 INFO:tasks.ceph.osd.7.smithi060.stderr:2024-09-13T09:10:27.392+0000 7fe539166640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 35343) UID: 0 2024-09-13T09:10:27.398 INFO:teuthology.orchestra.run.smithi031.stdout:All operations completed A-OK! 2024-09-13T09:10:27.443 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:27.443 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-13T09:10:27.452 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T09:10:27.469 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T09:10:27.470 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:27.470 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:27.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T09:10:27.640 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:27.641 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:27.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T09:10:27.780 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:27.781 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:27.795 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T09:10:27.922 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:27.922 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:27.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T09:10:28.055 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:28.055 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:28.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T09:10:28.197 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:28.198 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:28.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T09:10:28.333 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:28.333 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:28.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T09:10:28.476 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:28.476 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:28.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T09:10:28.615 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:28.616 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:28.630 DEBUG:teuthology.orchestra.run.smithi060:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T09:10:28.749 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-13T09:10:28.749 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T09:10:29.316 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T09:10:30.317 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-09-13T09:10:30.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T09:10:31.324 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-09-13T09:10:31.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T09:10:31.518 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-09-13T09:10:31.518 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:10:31.531 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-09-13T09:10:31.531 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:10:31.542 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-09-13T09:10:31.542 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T09:10:31.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T09:10:32.331 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T09:10:32.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T09:10:33.345 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T09:10:33.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T09:10:33.900 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T09:10:33.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T09:10:34.894 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T09:10:34.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T09:10:35.907 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T09:10:35.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T09:10:36.908 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T09:10:36.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T09:10:37.933 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T09:10:37.959 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T09:10:38.896 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T09:10:38.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T09:10:39.934 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T09:10:39.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T09:10:40.961 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T09:10:40.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T09:10:41.962 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T09:10:41.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T09:10:42.981 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T09:10:43.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T09:10:43.983 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T09:10:44.012 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T09:10:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T09:10:45.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T09:10:46.003 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T09:10:46.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-13T09:10:47.021 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T09:10:47.047 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T09:10:47.047 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T09:10:47.048 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T09:10:47.048 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T09:10:47.048 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T09:10:47.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:10:47.354 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:47.354 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":210,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:10:47.004896+0000","last_up_change":"2024-09-13T09:10:18.167310+0000","last_in_change":"2024-09-13T09:10:26.268217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:06:55.131330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:07:02.743102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:07:12.517158+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":99,"snap_epoch":190,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6818","nonce":3291756747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6822","nonce":3291756747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6830","nonce":3291756747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6826","nonce":3291756747}]},"public_addr":"172.21.15.31:6818/3291756747","cluster_addr":"172.21.15.31:6822/3291756747","heartbeat_back_addr":"172.21.15.31:6830/3291756747","heartbeat_front_addr":"172.21.15.31:6826/3291756747","state":["exists","up"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6801","nonce":533520566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6803","nonce":533520566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6807","nonce":533520566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6805","nonce":533520566}]},"public_addr":"172.21.15.31:6801/533520566","cluster_addr":"172.21.15.31:6803/533520566","heartbeat_back_addr":"172.21.15.31:6807/533520566","heartbeat_front_addr":"172.21.15.31:6805/533520566","state":["exists","up"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6819","nonce":2253604482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6823","nonce":2253604482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6829","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6831","nonce":2253604482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6825","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6827","nonce":2253604482}]},"public_addr":"172.21.15.31:6819/2253604482","cluster_addr":"172.21.15.31:6823/2253604482","heartbeat_back_addr":"172.21.15.31:6831/2253604482","heartbeat_front_addr":"172.21.15.31:6827/2253604482","state":["exists","up"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6809","nonce":3367214693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6811","nonce":3367214693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6815","nonce":3367214693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6813","nonce":3367214693}]},"public_addr":"172.21.15.31:6809/3367214693","cluster_addr":"172.21.15.31:6811/3367214693","heartbeat_back_addr":"172.21.15.31:6815/3367214693","heartbeat_front_addr":"172.21.15.31:6813/3367214693","state":["exists","up"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6801","nonce":3778074750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6803","nonce":3778074750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6807","nonce":3778074750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6805","nonce":3778074750}]},"public_addr":"172.21.15.60:6801/3778074750","cluster_addr":"172.21.15.60:6803/3778074750","heartbeat_back_addr":"172.21.15.60:6807/3778074750","heartbeat_front_addr":"172.21.15.60:6805/3778074750","state":["exists","up"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6809","nonce":1926603087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6813","nonce":1926603087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6824","nonce":1926603087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6817","nonce":1926603087}]},"public_addr":"172.21.15.60:6809/1926603087","cluster_addr":"172.21.15.60:6813/1926603087","heartbeat_back_addr":"172.21.15.60:6824/1926603087","heartbeat_front_addr":"172.21.15.60:6817/1926603087","state":["exists","up"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6821","nonce":766305500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6825","nonce":766305500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6831","nonce":766305500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6828","nonce":766305500}]},"public_addr":"172.21.15.60:6821/766305500","cluster_addr":"172.21.15.60:6825/766305500","heartbeat_back_addr":"172.21.15.60:6831/766305500","heartbeat_front_addr":"172.21.15.60:6828/766305500","state":["exists","up"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":157,"up_from":181,"up_thru":181,"down_at":158,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6812","nonce":3030128680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6816","nonce":3030128680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6829","nonce":3030128680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6822","nonce":3030128680}]},"public_addr":"172.21.15.60:6812/3030128680","cluster_addr":"172.21.15.60:6816/3030128680","heartbeat_back_addr":"172.21.15.60:6829/3030128680","heartbeat_front_addr":"172.21.15.60:6822/3030128680","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.183436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.264080+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.253587+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.244429+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.369458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.158089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.217295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.332254+0000","dead_epoch":158}],"pg_upmap":[{"pgid":"3.a","osds":[4,2]},{"pgid":"3.b","osds":[2,1]}],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":2,"to":7}]},{"pgid":"3.14","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:10:47.373 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T09:10:47.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:10:47.680 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:47.681 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":210,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:10:47.004896+0000","last_up_change":"2024-09-13T09:10:18.167310+0000","last_in_change":"2024-09-13T09:10:26.268217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:06:55.131330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:07:02.743102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:07:12.517158+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":99,"snap_epoch":190,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6818","nonce":3291756747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6822","nonce":3291756747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6830","nonce":3291756747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6826","nonce":3291756747}]},"public_addr":"172.21.15.31:6818/3291756747","cluster_addr":"172.21.15.31:6822/3291756747","heartbeat_back_addr":"172.21.15.31:6830/3291756747","heartbeat_front_addr":"172.21.15.31:6826/3291756747","state":["exists","up"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6801","nonce":533520566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6803","nonce":533520566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6807","nonce":533520566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6805","nonce":533520566}]},"public_addr":"172.21.15.31:6801/533520566","cluster_addr":"172.21.15.31:6803/533520566","heartbeat_back_addr":"172.21.15.31:6807/533520566","heartbeat_front_addr":"172.21.15.31:6805/533520566","state":["exists","up"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6819","nonce":2253604482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6823","nonce":2253604482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6829","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6831","nonce":2253604482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6825","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6827","nonce":2253604482}]},"public_addr":"172.21.15.31:6819/2253604482","cluster_addr":"172.21.15.31:6823/2253604482","heartbeat_back_addr":"172.21.15.31:6831/2253604482","heartbeat_front_addr":"172.21.15.31:6827/2253604482","state":["exists","up"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6809","nonce":3367214693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6811","nonce":3367214693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6815","nonce":3367214693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6813","nonce":3367214693}]},"public_addr":"172.21.15.31:6809/3367214693","cluster_addr":"172.21.15.31:6811/3367214693","heartbeat_back_addr":"172.21.15.31:6815/3367214693","heartbeat_front_addr":"172.21.15.31:6813/3367214693","state":["exists","up"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6801","nonce":3778074750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6803","nonce":3778074750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6807","nonce":3778074750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6805","nonce":3778074750}]},"public_addr":"172.21.15.60:6801/3778074750","cluster_addr":"172.21.15.60:6803/3778074750","heartbeat_back_addr":"172.21.15.60:6807/3778074750","heartbeat_front_addr":"172.21.15.60:6805/3778074750","state":["exists","up"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6809","nonce":1926603087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6813","nonce":1926603087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6824","nonce":1926603087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6817","nonce":1926603087}]},"public_addr":"172.21.15.60:6809/1926603087","cluster_addr":"172.21.15.60:6813/1926603087","heartbeat_back_addr":"172.21.15.60:6824/1926603087","heartbeat_front_addr":"172.21.15.60:6817/1926603087","state":["exists","up"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6821","nonce":766305500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6825","nonce":766305500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6831","nonce":766305500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6828","nonce":766305500}]},"public_addr":"172.21.15.60:6821/766305500","cluster_addr":"172.21.15.60:6825/766305500","heartbeat_back_addr":"172.21.15.60:6831/766305500","heartbeat_front_addr":"172.21.15.60:6828/766305500","state":["exists","up"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":157,"up_from":181,"up_thru":181,"down_at":158,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6812","nonce":3030128680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6816","nonce":3030128680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6829","nonce":3030128680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6822","nonce":3030128680}]},"public_addr":"172.21.15.60:6812/3030128680","cluster_addr":"172.21.15.60:6816/3030128680","heartbeat_back_addr":"172.21.15.60:6829/3030128680","heartbeat_front_addr":"172.21.15.60:6822/3030128680","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.183436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.264080+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.253587+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.244429+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.369458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.158089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.217295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.332254+0000","dead_epoch":158}],"pg_upmap":[{"pgid":"3.a","osds":[4,2]},{"pgid":"3.b","osds":[2,1]}],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":2,"to":7}]},{"pgid":"3.14","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:10:47.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T09:10:47.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T09:10:47.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T09:10:47.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T09:10:47.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T09:10:47.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T09:10:47.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T09:10:47.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T09:10:47.951 INFO:teuthology.orchestra.run.smithi031.stdout:64424509491 2024-09-13T09:10:47.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:10:47.966 INFO:teuthology.orchestra.run.smithi031.stdout:64424509491 2024-09-13T09:10:47.967 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:10:47.967 INFO:teuthology.orchestra.run.smithi031.stdout:777389080584 2024-09-13T09:10:47.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:10:47.978 INFO:teuthology.orchestra.run.smithi031.stdout:60129542194 2024-09-13T09:10:47.978 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:10:47.982 INFO:teuthology.orchestra.run.smithi031.stdout:64424509490 2024-09-13T09:10:47.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:10:47.984 INFO:teuthology.orchestra.run.smithi031.stdout:64424509489 2024-09-13T09:10:47.984 INFO:teuthology.orchestra.run.smithi031.stdout:60129542193 2024-09-13T09:10:47.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:10:47.985 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:10:47.986 INFO:teuthology.orchestra.run.smithi031.stdout:55834574898 2024-09-13T09:10:47.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:10:48.453 INFO:teuthology.orchestra.run.smithi031.stdout:64424509490 2024-09-13T09:10:48.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509490 for osd.0 2024-09-13T09:10:48.477 INFO:teuthology.orchestra.run.smithi031.stdout:777389080582 2024-09-13T09:10:48.499 INFO:teuthology.orchestra.run.smithi031.stdout:64424509490 2024-09-13T09:10:48.500 INFO:teuthology.orchestra.run.smithi031.stdout:60129542193 2024-09-13T09:10:48.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 777389080584 got 777389080582 for osd.7 2024-09-13T09:10:48.518 INFO:teuthology.orchestra.run.smithi031.stdout:55834574897 2024-09-13T09:10:48.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509490 for osd.1 2024-09-13T09:10:48.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542193 for osd.6 2024-09-13T09:10:48.529 INFO:teuthology.orchestra.run.smithi031.stdout:60129542192 2024-09-13T09:10:48.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574898 got 55834574897 for osd.5 2024-09-13T09:10:48.545 INFO:teuthology.orchestra.run.smithi031.stdout:64424509489 2024-09-13T09:10:48.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542192 for osd.4 2024-09-13T09:10:48.562 INFO:teuthology.orchestra.run.smithi031.stdout:64424509488 2024-09-13T09:10:48.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509490 got 64424509489 for osd.3 2024-09-13T09:10:48.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509489 got 64424509488 for osd.2 2024-09-13T09:10:49.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:10:49.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T09:10:49.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T09:10:49.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T09:10:49.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T09:10:49.556 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T09:10:49.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T09:10:49.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T09:10:49.997 INFO:teuthology.orchestra.run.smithi031.stdout:64424509490 2024-09-13T09:10:50.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509490 for osd.0 2024-09-13T09:10:50.044 INFO:teuthology.orchestra.run.smithi031.stdout:777389080584 2024-09-13T09:10:50.062 INFO:teuthology.orchestra.run.smithi031.stdout:64424509491 2024-09-13T09:10:50.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 777389080584 got 777389080584 for osd.7 2024-09-13T09:10:50.069 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:50.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509491 for osd.1 2024-09-13T09:10:50.080 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:50.083 INFO:teuthology.orchestra.run.smithi031.stdout:55834574899 2024-09-13T09:10:50.089 INFO:teuthology.orchestra.run.smithi031.stdout:64424509490 2024-09-13T09:10:50.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574898 got 55834574899 for osd.5 2024-09-13T09:10:50.103 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:50.111 INFO:teuthology.orchestra.run.smithi031.stdout:60129542193 2024-09-13T09:10:50.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509490 got 64424509490 for osd.3 2024-09-13T09:10:50.115 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:50.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542193 got 60129542193 for osd.4 2024-09-13T09:10:50.129 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:50.133 INFO:teuthology.orchestra.run.smithi031.stdout:60129542194 2024-09-13T09:10:50.149 INFO:teuthology.orchestra.run.smithi031.stdout:64424509489 2024-09-13T09:10:50.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542194 for osd.6 2024-09-13T09:10:50.150 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:50.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509489 got 64424509489 for osd.2 2024-09-13T09:10:50.166 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:51.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T09:10:51.322 INFO:teuthology.orchestra.run.smithi031.stdout:64424509491 2024-09-13T09:10:51.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509491 for osd.0 2024-09-13T09:10:51.341 DEBUG:teuthology.parallel:result is None 2024-09-13T09:10:51.341 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T09:10:51.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:10:51.635 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:51.636 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:10:51.652 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-13T09:10:51.112377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":336,"num_read":210966,"num_read_kb":690614,"num_write":25203,"num_write_kb":340526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21540,"ondisk_log_size":21540,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406268,"kb_used_data":20928,"kb_used_omap":30,"kb_used_meta":385185,"kb_avail":754568452,"statfs":{"total":773094113280,"available":772678094848,"internally_reserved":0,"allocated":21430272,"data_stored":10980674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31401,"internal_metadata":394429783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.124050"},"pg_stats":[{"pgid":"3.1f","version":"190'569","reported_seq":5735,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001506+0000","last_change":"2024-09-13T09:10:27.358289+0000","last_active":"2024-09-13T09:10:46.001506+0000","last_peered":"2024-09-13T09:10:46.001506+0000","last_clean":"2024-09-13T09:10:46.001506+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:46.001506+0000","last_undegraded":"2024-09-13T09:10:46.001506+0000","last_fullsized":"2024-09-13T09:10:46.001506+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016218700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5971,"num_read_kb":20218,"num_write":677,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"189'1226","reported_seq":5143,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001090+0000","last_change":"2024-09-13T09:10:27.357623+0000","last_active":"2024-09-13T09:10:46.001090+0000","last_peered":"2024-09-13T09:10:46.001090+0000","last_clean":"2024-09-13T09:10:46.001090+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:10:46.001090+0000","last_undegraded":"2024-09-13T09:10:46.001090+0000","last_fullsized":"2024-09-13T09:10:46.001090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082279900000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9326,"num_read_kb":19231,"num_write":1571,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"188'1409","reported_seq":5692,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935670+0000","last_change":"2024-09-13T09:10:37.937627+0000","last_active":"2024-09-13T09:10:46.935670+0000","last_peered":"2024-09-13T09:10:46.935670+0000","last_clean":"2024-09-13T09:10:46.935670+0000","last_became_active":"2024-09-13T09:10:37.936189+0000","last_became_peered":"2024-09-13T09:10:37.936189+0000","last_unstale":"2024-09-13T09:10:46.935670+0000","last_undegraded":"2024-09-13T09:10:46.935670+0000","last_fullsized":"2024-09-13T09:10:46.935670+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:43.640432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013860960000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8733,"num_read_kb":34149,"num_write":2234,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"190'732","reported_seq":4385,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435424+0000","last_change":"2024-09-13T09:10:36.914556+0000","last_active":"2024-09-13T09:10:46.435424+0000","last_peered":"2024-09-13T09:10:46.435424+0000","last_clean":"2024-09-13T09:10:46.435424+0000","last_became_active":"2024-09-13T09:10:36.913240+0000","last_became_peered":"2024-09-13T09:10:36.913240+0000","last_unstale":"2024-09-13T09:10:46.435424+0000","last_undegraded":"2024-09-13T09:10:46.435424+0000","last_fullsized":"2024-09-13T09:10:46.435424+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:19.776639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015845119999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5745,"num_read_kb":35631,"num_write":972,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"188'730","reported_seq":22464,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936232+0000","last_change":"2024-09-13T09:10:36.919210+0000","last_active":"2024-09-13T09:10:46.936232+0000","last_peered":"2024-09-13T09:10:46.936232+0000","last_clean":"2024-09-13T09:10:46.936232+0000","last_became_active":"2024-09-13T09:10:36.914093+0000","last_became_peered":"2024-09-13T09:10:36.914093+0000","last_unstale":"2024-09-13T09:10:46.936232+0000","last_undegraded":"2024-09-13T09:10:46.936232+0000","last_fullsized":"2024-09-13T09:10:46.936232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:46.181581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023266060000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":23676,"num_read_kb":48180,"num_write":787,"num_write_kb":18573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"188'704","reported_seq":6536,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435442+0000","last_change":"2024-09-13T09:10:27.300037+0000","last_active":"2024-09-13T09:10:46.435442+0000","last_peered":"2024-09-13T09:10:46.435442+0000","last_clean":"2024-09-13T09:10:46.435442+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:10:46.435442+0000","last_undegraded":"2024-09-13T09:10:46.435442+0000","last_fullsized":"2024-09-13T09:10:46.435442+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093814799999999998,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":8455,"num_read_kb":25867,"num_write":848,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"188'387","reported_seq":2116,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935630+0000","last_change":"2024-09-13T09:10:27.328428+0000","last_active":"2024-09-13T09:10:46.935630+0000","last_peered":"2024-09-13T09:10:46.935630+0000","last_clean":"2024-09-13T09:10:46.935630+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:10:46.935630+0000","last_undegraded":"2024-09-13T09:10:46.935630+0000","last_fullsized":"2024-09-13T09:10:46.935630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340506,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1827,"num_read_kb":8140,"num_write":348,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"189'714","reported_seq":3577,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001432+0000","last_change":"2024-09-13T09:10:27.358518+0000","last_active":"2024-09-13T09:10:46.001432+0000","last_peered":"2024-09-13T09:10:46.001432+0000","last_clean":"2024-09-13T09:10:46.001432+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:10:46.001432+0000","last_undegraded":"2024-09-13T09:10:46.001432+0000","last_fullsized":"2024-09-13T09:10:46.001432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016018569999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4805,"num_read_kb":14792,"num_write":763,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"189'668","reported_seq":5966,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001614+0000","last_change":"2024-09-13T09:10:27.358098+0000","last_active":"2024-09-13T09:10:46.001614+0000","last_peered":"2024-09-13T09:10:46.001614+0000","last_clean":"2024-09-13T09:10:46.001614+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:10:46.001614+0000","last_undegraded":"2024-09-13T09:10:46.001614+0000","last_fullsized":"2024-09-13T09:10:46.001614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689800000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7350,"num_read_kb":31391,"num_write":638,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"189'718","reported_seq":2129,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424676+0000","last_change":"2024-09-13T09:10:27.295495+0000","last_active":"2024-09-13T09:10:46.424676+0000","last_peered":"2024-09-13T09:10:46.424676+0000","last_clean":"2024-09-13T09:10:46.424676+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:46.424676+0000","last_undegraded":"2024-09-13T09:10:46.424676+0000","last_fullsized":"2024-09-13T09:10:46.424676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020123239999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2936,"num_read_kb":14217,"num_write":967,"num_write_kb":9811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"189'734","reported_seq":4409,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002493+0000","last_change":"2024-09-13T09:10:36.923247+0000","last_active":"2024-09-13T09:10:46.002493+0000","last_peered":"2024-09-13T09:10:46.002493+0000","last_clean":"2024-09-13T09:10:46.002493+0000","last_became_active":"2024-09-13T09:10:36.918448+0000","last_became_peered":"2024-09-13T09:10:36.918448+0000","last_unstale":"2024-09-13T09:10:46.002493+0000","last_undegraded":"2024-09-13T09:10:46.002493+0000","last_fullsized":"2024-09-13T09:10:46.002493+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:36.185691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021014559999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5305,"num_read_kb":24795,"num_write":695,"num_write_kb":16167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"190'725","reported_seq":19435,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435181+0000","last_change":"2024-09-13T09:10:27.300467+0000","last_active":"2024-09-13T09:10:46.435181+0000","last_peered":"2024-09-13T09:10:46.435181+0000","last_clean":"2024-09-13T09:10:46.435181+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:10:46.435181+0000","last_undegraded":"2024-09-13T09:10:46.435181+0000","last_fullsized":"2024-09-13T09:10:46.435181+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012366219999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21128,"num_read_kb":34179,"num_write":635,"num_write_kb":10198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":221,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:10:33.349791+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:10:33.349249+0000","last_became_peered":"2024-09-13T09:10:33.349249+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:24.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"190'700","reported_seq":6003,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935995+0000","last_change":"2024-09-13T09:10:27.329077+0000","last_active":"2024-09-13T09:10:46.935995+0000","last_peered":"2024-09-13T09:10:46.935995+0000","last_clean":"2024-09-13T09:10:46.935995+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:10:46.935995+0000","last_undegraded":"2024-09-13T09:10:46.935995+0000","last_fullsized":"2024-09-13T09:10:46.935995+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:13.066567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028872860000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6169,"num_read_kb":21942,"num_write":789,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936143+0000","last_change":"2024-09-13T09:10:33.349800+0000","last_active":"2024-09-13T09:10:46.936143+0000","last_peered":"2024-09-13T09:10:46.936143+0000","last_clean":"2024-09-13T09:10:46.936143+0000","last_became_active":"2024-09-13T09:10:33.349192+0000","last_became_peered":"2024-09-13T09:10:33.349192+0000","last_unstale":"2024-09-13T09:10:46.936143+0000","last_undegraded":"2024-09-13T09:10:46.936143+0000","last_fullsized":"2024-09-13T09:10:46.936143+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:16.106956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"189'433","reported_seq":2543,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936105+0000","last_change":"2024-09-13T09:10:27.327294+0000","last_active":"2024-09-13T09:10:46.936105+0000","last_peered":"2024-09-13T09:10:46.936105+0000","last_clean":"2024-09-13T09:10:46.936105+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:10:46.936105+0000","last_undegraded":"2024-09-13T09:10:46.936105+0000","last_fullsized":"2024-09-13T09:10:46.936105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001256109,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2732,"num_read_kb":14647,"num_write":388,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"188'621","reported_seq":6246,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001146+0000","last_change":"2024-09-13T09:10:27.286574+0000","last_active":"2024-09-13T09:10:46.001146+0000","last_peered":"2024-09-13T09:10:46.001146+0000","last_clean":"2024-09-13T09:10:46.001146+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:10:46.001146+0000","last_undegraded":"2024-09-13T09:10:46.001146+0000","last_fullsized":"2024-09-13T09:10:46.001146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043649599999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6810,"num_read_kb":26019,"num_write":814,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001342+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:10:46.001342+0000","last_peered":"2024-09-13T09:10:46.001342+0000","last_clean":"2024-09-13T09:10:46.001342+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:10:46.001342+0000","last_undegraded":"2024-09-13T09:10:46.001342+0000","last_fullsized":"2024-09-13T09:10:46.001342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"189'717","reported_seq":3887,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424566+0000","last_change":"2024-09-13T09:10:27.294511+0000","last_active":"2024-09-13T09:10:46.424566+0000","last_peered":"2024-09-13T09:10:46.424566+0000","last_clean":"2024-09-13T09:10:46.424566+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:10:46.424566+0000","last_undegraded":"2024-09-13T09:10:46.424566+0000","last_fullsized":"2024-09-13T09:10:46.424566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014721929999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4939,"num_read_kb":19757,"num_write":904,"num_write_kb":10484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"190'667","reported_seq":3539,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002172+0000","last_change":"2024-09-13T09:10:36.919578+0000","last_active":"2024-09-13T09:10:46.002172+0000","last_peered":"2024-09-13T09:10:46.002172+0000","last_clean":"2024-09-13T09:10:46.002172+0000","last_became_active":"2024-09-13T09:10:36.917124+0000","last_became_peered":"2024-09-13T09:10:36.917124+0000","last_unstale":"2024-09-13T09:10:46.002172+0000","last_undegraded":"2024-09-13T09:10:46.002172+0000","last_fullsized":"2024-09-13T09:10:46.002172+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:11.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011757899999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3513,"num_read_kb":18416,"num_write":638,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":207,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935660+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:10:46.935660+0000","last_peered":"2024-09-13T09:10:46.935660+0000","last_clean":"2024-09-13T09:10:46.935660+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:10:46.935660+0000","last_undegraded":"2024-09-13T09:10:46.935660+0000","last_fullsized":"2024-09-13T09:10:46.935660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"189'504","reported_seq":3792,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001037+0000","last_change":"2024-09-13T09:10:27.357629+0000","last_active":"2024-09-13T09:10:46.001037+0000","last_peered":"2024-09-13T09:10:46.001037+0000","last_clean":"2024-09-13T09:10:46.001037+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:10:46.001037+0000","last_undegraded":"2024-09-13T09:10:46.001037+0000","last_fullsized":"2024-09-13T09:10:46.001037+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:27.395379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095688200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3585,"num_read_kb":19320,"num_write":505,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935744+0000","last_change":"2024-09-13T09:10:19.191965+0000","last_active":"2024-09-13T09:10:46.935744+0000","last_peered":"2024-09-13T09:10:46.935744+0000","last_clean":"2024-09-13T09:10:46.935744+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:46.935744+0000","last_undegraded":"2024-09-13T09:10:46.935744+0000","last_fullsized":"2024-09-13T09:10:46.935744+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:17.687266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"190'422","reported_seq":2354,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000648+0000","last_change":"2024-09-13T09:10:27.286775+0000","last_active":"2024-09-13T09:10:46.000648+0000","last_peered":"2024-09-13T09:10:46.000648+0000","last_clean":"2024-09-13T09:10:46.000648+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:10:46.000648+0000","last_undegraded":"2024-09-13T09:10:46.000648+0000","last_fullsized":"2024-09-13T09:10:46.000648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074152199999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2071,"num_read_kb":14898,"num_write":412,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935439+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:10:46.935439+0000","last_peered":"2024-09-13T09:10:46.935439+0000","last_clean":"2024-09-13T09:10:46.935439+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:10:46.935439+0000","last_undegraded":"2024-09-13T09:10:46.935439+0000","last_fullsized":"2024-09-13T09:10:46.935439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"189'405","reported_seq":2579,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936353+0000","last_change":"2024-09-13T09:10:27.326747+0000","last_active":"2024-09-13T09:10:46.936353+0000","last_peered":"2024-09-13T09:10:46.936353+0000","last_clean":"2024-09-13T09:10:46.936353+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:10:46.936353+0000","last_undegraded":"2024-09-13T09:10:46.936353+0000","last_fullsized":"2024-09-13T09:10:46.936353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037313,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2540,"num_read_kb":12233,"num_write":423,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935747+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:10:46.935747+0000","last_peered":"2024-09-13T09:10:46.935747+0000","last_clean":"2024-09-13T09:10:46.935747+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:10:46.935747+0000","last_undegraded":"2024-09-13T09:10:46.935747+0000","last_fullsized":"2024-09-13T09:10:46.935747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":335,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435002+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:10:46.435002+0000","last_peered":"2024-09-13T09:10:46.435002+0000","last_clean":"2024-09-13T09:10:46.435002+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:10:46.435002+0000","last_undegraded":"2024-09-13T09:10:46.435002+0000","last_fullsized":"2024-09-13T09:10:46.435002+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"190'881","reported_seq":6152,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935741+0000","last_change":"2024-09-13T09:10:27.327530+0000","last_active":"2024-09-13T09:10:46.935741+0000","last_peered":"2024-09-13T09:10:46.935741+0000","last_clean":"2024-09-13T09:10:46.935741+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:10:46.935741+0000","last_undegraded":"2024-09-13T09:10:46.935741+0000","last_fullsized":"2024-09-13T09:10:46.935741+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:33.786643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018329830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6854,"num_read_kb":23202,"num_write":1277,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"189'594","reported_seq":3645,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935677+0000","last_change":"2024-09-13T09:10:27.288793+0000","last_active":"2024-09-13T09:10:46.935677+0000","last_peered":"2024-09-13T09:10:46.935677+0000","last_clean":"2024-09-13T09:10:46.935677+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:46.935677+0000","last_undegraded":"2024-09-13T09:10:46.935677+0000","last_fullsized":"2024-09-13T09:10:46.935677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099698400000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4866,"num_read_kb":16848,"num_write":688,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"188'590","reported_seq":3110,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936333+0000","last_change":"2024-09-13T09:10:27.326271+0000","last_active":"2024-09-13T09:10:46.936333+0000","last_peered":"2024-09-13T09:10:46.936333+0000","last_clean":"2024-09-13T09:10:46.936333+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:46.936333+0000","last_undegraded":"2024-09-13T09:10:46.936333+0000","last_fullsized":"2024-09-13T09:10:46.936333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074287699999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4001,"num_read_kb":22630,"num_write":547,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"188'428","reported_seq":3878,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424141+0000","last_change":"2024-09-13T09:10:27.294587+0000","last_active":"2024-09-13T09:10:46.424141+0000","last_peered":"2024-09-13T09:10:46.424141+0000","last_clean":"2024-09-13T09:10:46.424141+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:10:46.424141+0000","last_undegraded":"2024-09-13T09:10:46.424141+0000","last_fullsized":"2024-09-13T09:10:46.424141+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:11.164172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013662279999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3594,"num_read_kb":33920,"num_write":421,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"187'360","reported_seq":17058,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002334+0000","last_change":"2024-09-13T09:10:35.921897+0000","last_active":"2024-09-13T09:10:46.002334+0000","last_peered":"2024-09-13T09:10:46.002334+0000","last_clean":"2024-09-13T09:10:46.002334+0000","last_became_active":"2024-09-13T09:10:35.919376+0000","last_became_peered":"2024-09-13T09:10:35.919376+0000","last_unstale":"2024-09-13T09:10:46.002334+0000","last_undegraded":"2024-09-13T09:10:46.002334+0000","last_fullsized":"2024-09-13T09:10:46.002334+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:08.251141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088103200000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":16596,"num_read_kb":27933,"num_write":361,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"190'410","reported_seq":3137,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935350+0000","last_change":"2024-09-13T09:10:27.289763+0000","last_active":"2024-09-13T09:10:46.935350+0000","last_peered":"2024-09-13T09:10:46.935350+0000","last_clean":"2024-09-13T09:10:46.935350+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:46.935350+0000","last_undegraded":"2024-09-13T09:10:46.935350+0000","last_fullsized":"2024-09-13T09:10:46.935350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001690069,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3095,"num_read_kb":9820,"num_write":360,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"190'884","reported_seq":7372,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424756+0000","last_change":"2024-09-13T09:10:35.916276+0000","last_active":"2024-09-13T09:10:46.424756+0000","last_peered":"2024-09-13T09:10:46.424756+0000","last_clean":"2024-09-13T09:10:46.424756+0000","last_became_active":"2024-09-13T09:10:35.914456+0000","last_became_peered":"2024-09-13T09:10:35.914456+0000","last_unstale":"2024-09-13T09:10:46.424756+0000","last_undegraded":"2024-09-13T09:10:46.424756+0000","last_fullsized":"2024-09-13T09:10:46.424756+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:13.805979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001663692,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9292,"num_read_kb":23776,"num_write":1223,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"189'477","reported_seq":5178,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002164+0000","last_change":"2024-09-13T09:10:35.918190+0000","last_active":"2024-09-13T09:10:46.002164+0000","last_peered":"2024-09-13T09:10:46.002164+0000","last_clean":"2024-09-13T09:10:46.002164+0000","last_became_active":"2024-09-13T09:10:35.915413+0000","last_became_peered":"2024-09-13T09:10:35.915413+0000","last_unstale":"2024-09-13T09:10:46.002164+0000","last_undegraded":"2024-09-13T09:10:46.002164+0000","last_fullsized":"2024-09-13T09:10:46.002164+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:23.432147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001550994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5016,"num_read_kb":17262,"num_write":425,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"188'518","reported_seq":2907,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435248+0000","last_change":"2024-09-13T09:10:27.300360+0000","last_active":"2024-09-13T09:10:46.435248+0000","last_peered":"2024-09-13T09:10:46.435248+0000","last_clean":"2024-09-13T09:10:46.435248+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:10:46.435248+0000","last_undegraded":"2024-09-13T09:10:46.435248+0000","last_fullsized":"2024-09-13T09:10:46.435248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011980339999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2903,"num_read_kb":17169,"num_write":491,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"190'1187","reported_seq":4411,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935485+0000","last_change":"2024-09-13T09:10:27.289049+0000","last_active":"2024-09-13T09:10:46.935485+0000","last_peered":"2024-09-13T09:10:46.935485+0000","last_clean":"2024-09-13T09:10:46.935485+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:46.935485+0000","last_undegraded":"2024-09-13T09:10:46.935485+0000","last_fullsized":"2024-09-13T09:10:46.935485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001456372,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7816,"num_read_kb":12367,"num_write":1664,"num_write_kb":5387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"189'718","reported_seq":4615,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000641+0000","last_change":"2024-09-13T09:10:27.287499+0000","last_active":"2024-09-13T09:10:46.000641+0000","last_peered":"2024-09-13T09:10:46.000641+0000","last_clean":"2024-09-13T09:10:46.000641+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:46.000641+0000","last_undegraded":"2024-09-13T09:10:46.000641+0000","last_fullsized":"2024-09-13T09:10:46.000641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012075230000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5826,"num_read_kb":14063,"num_write":971,"num_write_kb":11379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"187'588","reported_seq":4153,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935575+0000","last_change":"2024-09-13T09:10:31.329333+0000","last_active":"2024-09-13T09:10:46.935575+0000","last_peered":"2024-09-13T09:10:46.935575+0000","last_clean":"2024-09-13T09:10:46.935575+0000","last_became_active":"2024-09-13T09:10:31.325692+0000","last_became_peered":"2024-09-13T09:10:31.325692+0000","last_unstale":"2024-09-13T09:10:46.935575+0000","last_undegraded":"2024-09-13T09:10:46.935575+0000","last_fullsized":"2024-09-13T09:10:46.935575+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:57.571689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015796709999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3375,"num_read_kb":13502,"num_write":606,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":336,"num_read":210850,"num_read_kb":690514,"num_write":25042,"num_write_kb":338108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3692,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":21420,"ondisk_log_size":21420,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080584,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":2472,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65584,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306256,"statfs":{"total":96636764160,"available":96569606144,"internally_reserved":0,"allocated":2523136,"data_stored":1220555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64616904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.22,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.875,"5min":1.361,"15min":1.361},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63500000000000001,"15min":0.64700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.1459999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.747}]},{"osd":1,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63600000000000001},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.88100000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":1.444,"15min":1.444},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1850000000000001,"5min":1.766,"15min":1.766},"last":0.88}]},{"osd":4,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2170000000000001,"5min":1.343,"15min":1.343},"last":1.137}]},{"osd":5,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.919,"15min":1.919},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.90400000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4019999999999999,"5min":1.867,"15min":1.867},"last":1.081},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.238}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.2789999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69676,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":2519040,"data_stored":1220536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.67300000000000004,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.226,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73099999999999998,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.024,"5min":1.353,"15min":1.353},"last":0.504}]},{"osd":1,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.626},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.2350000000000001,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.401,"5min":1.681,"15min":1.681},"last":0.437},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.3,"5min":1.401,"15min":1.401},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":1.0249999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.288,"5min":1.998,"15min":1.998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.391,"5min":1.651,"15min":1.651},"last":0.879}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4379999999999999,"5min":1.506,"15min":1.506},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0009999999999999,"15min":1.0449999999999999},"min":{"1min":0.48899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.571,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.071},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3500000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.089,"15min":1.125},"min":{"1min":0.621,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.417,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":0.83599999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56316,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315524,"statfs":{"total":96636764160,"available":96579096576,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.30199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.86399999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.628}]},{"osd":2,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.624,"15min":0.64400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.054},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.90500000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.72399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0900000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3480000000000001,"5min":1.407,"15min":1.407},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3280000000000001,"5min":1.415,"15min":1.415},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.335,"5min":1.448,"15min":1.448},"last":1.145}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94399999999999995,"15min":0.97899999999999998},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.268},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1539999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.228}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93100000000000005,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4510000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.111},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.05},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.49,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.369},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.3029999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33588,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338252,"statfs":{"total":96636764160,"available":96602370048,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.59599999999999997},"min":{"1min":0.317,"5min":0.245,"15min":0.245},"max":{"1min":0.98099999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.625},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.379}]},{"osd":2,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66000000000000003,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.79200000000000004,"15min":0.86699999999999999},"min":{"1min":0.38500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.149,"5min":3.855,"15min":3.855},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0309999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.432},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.82999999999999996,"15min":0.92600000000000005},"min":{"1min":0.40999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.341,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.58899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":0.93799999999999994},"min":{"1min":0.54800000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.218,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.63}]},{"osd":5,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.94099999999999995,"15min":0.97399999999999998},"min":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2949999999999999,"5min":2.532,"15min":2.532},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.0629999999999999,"15min":1.123},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.516,"5min":2.181,"15min":2.181},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.056,"15min":1.1240000000000001},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3700000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.018,"15min":1.073},"min":{"1min":0.63100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4910000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:08:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.528,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0840000000000001,"15min":1.1220000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542195,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37692,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334148,"statfs":{"total":96636764160,"available":96598167552,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.434,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1759999999999999,"5min":1.552,"15min":1.552},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.446,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.752}]},{"osd":1,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.79600000000000004},"min":{"1min":0.47799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2989999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":1.097},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.92000000000000004},"min":{"1min":0.53200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2669999999999999,"5min":2.706,"15min":2.706},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.93600000000000005,"15min":0.89700000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.649,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.143},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.58899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1439999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.93400000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.022},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.92200000000000004,"15min":0.96399999999999997},"min":{"1min":0.498,"5min":0.377,"15min":0.377},"max":{"1min":1.5489999999999999,"5min":1.7,"15min":1.7},"last":1.202}]},{"osd":5,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.046,"15min":1.0800000000000001},"min":{"1min":0.5,"5min":0.497,"15min":0.497},"max":{"1min":1.359,"5min":1.776,"15min":1.776},"last":1.266},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.04,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.6220000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.323}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.4139999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3520000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542194,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.121,"5min":2.649,"15min":2.649},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.202,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1819999999999999,"5min":1.3,"15min":1.3},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.88,"15min":0.83099999999999996},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.996},{"interface":"front","average":{"1min":1.042,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.03}]},{"osd":3,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.91600000000000004,"15min":0.86799999999999999},"min":{"1min":0.623,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.591,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.149},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.98999999999999999,"15min":0.98199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.90100000000000002,"15min":0.85799999999999998},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.008,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.95899999999999996,"15min":0.89000000000000001},"min":{"1min":0.54900000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.74,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.2450000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.98599999999999999,"15min":0.94699999999999995},"min":{"1min":0.44600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.042,"15min":1.0389999999999999},"min":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.21}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.284}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.032,"15min":1.1499999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1930000000000001,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":1.194}]},{"osd":1,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.83599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2490000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.335,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":1.0269999999999999,"15min":1.1319999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2809999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.1990000000000001},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3799999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":1.145},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.151,"5min":1.585,"15min":1.585},"last":1.034}]},{"osd":4,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.997,"15min":1.024},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4119999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.321},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.25,"15min":1.3799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.351,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.417}]},{"osd":6,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.97999999999999998,"15min":0.98399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.278,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.355,"5min":1.899,"15min":1.899},"last":1.518}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.5509999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:10:51.655 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:10:51.995 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:51.996 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:10:52.015 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-13T09:10:51.112377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":336,"num_read":210966,"num_read_kb":690614,"num_write":25203,"num_write_kb":340526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21540,"ondisk_log_size":21540,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406268,"kb_used_data":20928,"kb_used_omap":30,"kb_used_meta":385185,"kb_avail":754568452,"statfs":{"total":773094113280,"available":772678094848,"internally_reserved":0,"allocated":21430272,"data_stored":10980674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31401,"internal_metadata":394429783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.124050"},"pg_stats":[{"pgid":"3.1f","version":"190'569","reported_seq":5735,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001506+0000","last_change":"2024-09-13T09:10:27.358289+0000","last_active":"2024-09-13T09:10:46.001506+0000","last_peered":"2024-09-13T09:10:46.001506+0000","last_clean":"2024-09-13T09:10:46.001506+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:46.001506+0000","last_undegraded":"2024-09-13T09:10:46.001506+0000","last_fullsized":"2024-09-13T09:10:46.001506+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016218700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5971,"num_read_kb":20218,"num_write":677,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"189'1226","reported_seq":5143,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001090+0000","last_change":"2024-09-13T09:10:27.357623+0000","last_active":"2024-09-13T09:10:46.001090+0000","last_peered":"2024-09-13T09:10:46.001090+0000","last_clean":"2024-09-13T09:10:46.001090+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:10:46.001090+0000","last_undegraded":"2024-09-13T09:10:46.001090+0000","last_fullsized":"2024-09-13T09:10:46.001090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082279900000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9326,"num_read_kb":19231,"num_write":1571,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"188'1409","reported_seq":5692,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935670+0000","last_change":"2024-09-13T09:10:37.937627+0000","last_active":"2024-09-13T09:10:46.935670+0000","last_peered":"2024-09-13T09:10:46.935670+0000","last_clean":"2024-09-13T09:10:46.935670+0000","last_became_active":"2024-09-13T09:10:37.936189+0000","last_became_peered":"2024-09-13T09:10:37.936189+0000","last_unstale":"2024-09-13T09:10:46.935670+0000","last_undegraded":"2024-09-13T09:10:46.935670+0000","last_fullsized":"2024-09-13T09:10:46.935670+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:43.640432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013860960000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8733,"num_read_kb":34149,"num_write":2234,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"190'732","reported_seq":4385,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435424+0000","last_change":"2024-09-13T09:10:36.914556+0000","last_active":"2024-09-13T09:10:46.435424+0000","last_peered":"2024-09-13T09:10:46.435424+0000","last_clean":"2024-09-13T09:10:46.435424+0000","last_became_active":"2024-09-13T09:10:36.913240+0000","last_became_peered":"2024-09-13T09:10:36.913240+0000","last_unstale":"2024-09-13T09:10:46.435424+0000","last_undegraded":"2024-09-13T09:10:46.435424+0000","last_fullsized":"2024-09-13T09:10:46.435424+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:19.776639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015845119999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5745,"num_read_kb":35631,"num_write":972,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"188'730","reported_seq":22464,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936232+0000","last_change":"2024-09-13T09:10:36.919210+0000","last_active":"2024-09-13T09:10:46.936232+0000","last_peered":"2024-09-13T09:10:46.936232+0000","last_clean":"2024-09-13T09:10:46.936232+0000","last_became_active":"2024-09-13T09:10:36.914093+0000","last_became_peered":"2024-09-13T09:10:36.914093+0000","last_unstale":"2024-09-13T09:10:46.936232+0000","last_undegraded":"2024-09-13T09:10:46.936232+0000","last_fullsized":"2024-09-13T09:10:46.936232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:46.181581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023266060000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":23676,"num_read_kb":48180,"num_write":787,"num_write_kb":18573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"188'704","reported_seq":6536,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435442+0000","last_change":"2024-09-13T09:10:27.300037+0000","last_active":"2024-09-13T09:10:46.435442+0000","last_peered":"2024-09-13T09:10:46.435442+0000","last_clean":"2024-09-13T09:10:46.435442+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:10:46.435442+0000","last_undegraded":"2024-09-13T09:10:46.435442+0000","last_fullsized":"2024-09-13T09:10:46.435442+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093814799999999998,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":8455,"num_read_kb":25867,"num_write":848,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"188'387","reported_seq":2116,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935630+0000","last_change":"2024-09-13T09:10:27.328428+0000","last_active":"2024-09-13T09:10:46.935630+0000","last_peered":"2024-09-13T09:10:46.935630+0000","last_clean":"2024-09-13T09:10:46.935630+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:10:46.935630+0000","last_undegraded":"2024-09-13T09:10:46.935630+0000","last_fullsized":"2024-09-13T09:10:46.935630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340506,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1827,"num_read_kb":8140,"num_write":348,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"189'714","reported_seq":3577,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001432+0000","last_change":"2024-09-13T09:10:27.358518+0000","last_active":"2024-09-13T09:10:46.001432+0000","last_peered":"2024-09-13T09:10:46.001432+0000","last_clean":"2024-09-13T09:10:46.001432+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:10:46.001432+0000","last_undegraded":"2024-09-13T09:10:46.001432+0000","last_fullsized":"2024-09-13T09:10:46.001432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016018569999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4805,"num_read_kb":14792,"num_write":763,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"189'668","reported_seq":5966,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001614+0000","last_change":"2024-09-13T09:10:27.358098+0000","last_active":"2024-09-13T09:10:46.001614+0000","last_peered":"2024-09-13T09:10:46.001614+0000","last_clean":"2024-09-13T09:10:46.001614+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:10:46.001614+0000","last_undegraded":"2024-09-13T09:10:46.001614+0000","last_fullsized":"2024-09-13T09:10:46.001614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689800000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7350,"num_read_kb":31391,"num_write":638,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"189'718","reported_seq":2129,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424676+0000","last_change":"2024-09-13T09:10:27.295495+0000","last_active":"2024-09-13T09:10:46.424676+0000","last_peered":"2024-09-13T09:10:46.424676+0000","last_clean":"2024-09-13T09:10:46.424676+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:46.424676+0000","last_undegraded":"2024-09-13T09:10:46.424676+0000","last_fullsized":"2024-09-13T09:10:46.424676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020123239999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2936,"num_read_kb":14217,"num_write":967,"num_write_kb":9811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"189'734","reported_seq":4409,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002493+0000","last_change":"2024-09-13T09:10:36.923247+0000","last_active":"2024-09-13T09:10:46.002493+0000","last_peered":"2024-09-13T09:10:46.002493+0000","last_clean":"2024-09-13T09:10:46.002493+0000","last_became_active":"2024-09-13T09:10:36.918448+0000","last_became_peered":"2024-09-13T09:10:36.918448+0000","last_unstale":"2024-09-13T09:10:46.002493+0000","last_undegraded":"2024-09-13T09:10:46.002493+0000","last_fullsized":"2024-09-13T09:10:46.002493+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:36.185691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021014559999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5305,"num_read_kb":24795,"num_write":695,"num_write_kb":16167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"190'725","reported_seq":19435,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435181+0000","last_change":"2024-09-13T09:10:27.300467+0000","last_active":"2024-09-13T09:10:46.435181+0000","last_peered":"2024-09-13T09:10:46.435181+0000","last_clean":"2024-09-13T09:10:46.435181+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:10:46.435181+0000","last_undegraded":"2024-09-13T09:10:46.435181+0000","last_fullsized":"2024-09-13T09:10:46.435181+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012366219999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21128,"num_read_kb":34179,"num_write":635,"num_write_kb":10198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":221,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:10:33.349791+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:10:33.349249+0000","last_became_peered":"2024-09-13T09:10:33.349249+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:24.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"190'700","reported_seq":6003,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935995+0000","last_change":"2024-09-13T09:10:27.329077+0000","last_active":"2024-09-13T09:10:46.935995+0000","last_peered":"2024-09-13T09:10:46.935995+0000","last_clean":"2024-09-13T09:10:46.935995+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:10:46.935995+0000","last_undegraded":"2024-09-13T09:10:46.935995+0000","last_fullsized":"2024-09-13T09:10:46.935995+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:13.066567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028872860000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6169,"num_read_kb":21942,"num_write":789,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936143+0000","last_change":"2024-09-13T09:10:33.349800+0000","last_active":"2024-09-13T09:10:46.936143+0000","last_peered":"2024-09-13T09:10:46.936143+0000","last_clean":"2024-09-13T09:10:46.936143+0000","last_became_active":"2024-09-13T09:10:33.349192+0000","last_became_peered":"2024-09-13T09:10:33.349192+0000","last_unstale":"2024-09-13T09:10:46.936143+0000","last_undegraded":"2024-09-13T09:10:46.936143+0000","last_fullsized":"2024-09-13T09:10:46.936143+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:16.106956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"189'433","reported_seq":2543,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936105+0000","last_change":"2024-09-13T09:10:27.327294+0000","last_active":"2024-09-13T09:10:46.936105+0000","last_peered":"2024-09-13T09:10:46.936105+0000","last_clean":"2024-09-13T09:10:46.936105+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:10:46.936105+0000","last_undegraded":"2024-09-13T09:10:46.936105+0000","last_fullsized":"2024-09-13T09:10:46.936105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001256109,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2732,"num_read_kb":14647,"num_write":388,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"188'621","reported_seq":6246,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001146+0000","last_change":"2024-09-13T09:10:27.286574+0000","last_active":"2024-09-13T09:10:46.001146+0000","last_peered":"2024-09-13T09:10:46.001146+0000","last_clean":"2024-09-13T09:10:46.001146+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:10:46.001146+0000","last_undegraded":"2024-09-13T09:10:46.001146+0000","last_fullsized":"2024-09-13T09:10:46.001146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043649599999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6810,"num_read_kb":26019,"num_write":814,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001342+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:10:46.001342+0000","last_peered":"2024-09-13T09:10:46.001342+0000","last_clean":"2024-09-13T09:10:46.001342+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:10:46.001342+0000","last_undegraded":"2024-09-13T09:10:46.001342+0000","last_fullsized":"2024-09-13T09:10:46.001342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"189'717","reported_seq":3887,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424566+0000","last_change":"2024-09-13T09:10:27.294511+0000","last_active":"2024-09-13T09:10:46.424566+0000","last_peered":"2024-09-13T09:10:46.424566+0000","last_clean":"2024-09-13T09:10:46.424566+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:10:46.424566+0000","last_undegraded":"2024-09-13T09:10:46.424566+0000","last_fullsized":"2024-09-13T09:10:46.424566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014721929999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4939,"num_read_kb":19757,"num_write":904,"num_write_kb":10484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"190'667","reported_seq":3539,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002172+0000","last_change":"2024-09-13T09:10:36.919578+0000","last_active":"2024-09-13T09:10:46.002172+0000","last_peered":"2024-09-13T09:10:46.002172+0000","last_clean":"2024-09-13T09:10:46.002172+0000","last_became_active":"2024-09-13T09:10:36.917124+0000","last_became_peered":"2024-09-13T09:10:36.917124+0000","last_unstale":"2024-09-13T09:10:46.002172+0000","last_undegraded":"2024-09-13T09:10:46.002172+0000","last_fullsized":"2024-09-13T09:10:46.002172+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:11.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011757899999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3513,"num_read_kb":18416,"num_write":638,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":207,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935660+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:10:46.935660+0000","last_peered":"2024-09-13T09:10:46.935660+0000","last_clean":"2024-09-13T09:10:46.935660+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:10:46.935660+0000","last_undegraded":"2024-09-13T09:10:46.935660+0000","last_fullsized":"2024-09-13T09:10:46.935660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"189'504","reported_seq":3792,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001037+0000","last_change":"2024-09-13T09:10:27.357629+0000","last_active":"2024-09-13T09:10:46.001037+0000","last_peered":"2024-09-13T09:10:46.001037+0000","last_clean":"2024-09-13T09:10:46.001037+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:10:46.001037+0000","last_undegraded":"2024-09-13T09:10:46.001037+0000","last_fullsized":"2024-09-13T09:10:46.001037+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:27.395379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095688200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3585,"num_read_kb":19320,"num_write":505,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935744+0000","last_change":"2024-09-13T09:10:19.191965+0000","last_active":"2024-09-13T09:10:46.935744+0000","last_peered":"2024-09-13T09:10:46.935744+0000","last_clean":"2024-09-13T09:10:46.935744+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:46.935744+0000","last_undegraded":"2024-09-13T09:10:46.935744+0000","last_fullsized":"2024-09-13T09:10:46.935744+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:17.687266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"190'422","reported_seq":2354,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000648+0000","last_change":"2024-09-13T09:10:27.286775+0000","last_active":"2024-09-13T09:10:46.000648+0000","last_peered":"2024-09-13T09:10:46.000648+0000","last_clean":"2024-09-13T09:10:46.000648+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:10:46.000648+0000","last_undegraded":"2024-09-13T09:10:46.000648+0000","last_fullsized":"2024-09-13T09:10:46.000648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074152199999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2071,"num_read_kb":14898,"num_write":412,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935439+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:10:46.935439+0000","last_peered":"2024-09-13T09:10:46.935439+0000","last_clean":"2024-09-13T09:10:46.935439+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:10:46.935439+0000","last_undegraded":"2024-09-13T09:10:46.935439+0000","last_fullsized":"2024-09-13T09:10:46.935439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"189'405","reported_seq":2579,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936353+0000","last_change":"2024-09-13T09:10:27.326747+0000","last_active":"2024-09-13T09:10:46.936353+0000","last_peered":"2024-09-13T09:10:46.936353+0000","last_clean":"2024-09-13T09:10:46.936353+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:10:46.936353+0000","last_undegraded":"2024-09-13T09:10:46.936353+0000","last_fullsized":"2024-09-13T09:10:46.936353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037313,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2540,"num_read_kb":12233,"num_write":423,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935747+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:10:46.935747+0000","last_peered":"2024-09-13T09:10:46.935747+0000","last_clean":"2024-09-13T09:10:46.935747+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:10:46.935747+0000","last_undegraded":"2024-09-13T09:10:46.935747+0000","last_fullsized":"2024-09-13T09:10:46.935747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":335,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435002+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:10:46.435002+0000","last_peered":"2024-09-13T09:10:46.435002+0000","last_clean":"2024-09-13T09:10:46.435002+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:10:46.435002+0000","last_undegraded":"2024-09-13T09:10:46.435002+0000","last_fullsized":"2024-09-13T09:10:46.435002+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"190'881","reported_seq":6152,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935741+0000","last_change":"2024-09-13T09:10:27.327530+0000","last_active":"2024-09-13T09:10:46.935741+0000","last_peered":"2024-09-13T09:10:46.935741+0000","last_clean":"2024-09-13T09:10:46.935741+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:10:46.935741+0000","last_undegraded":"2024-09-13T09:10:46.935741+0000","last_fullsized":"2024-09-13T09:10:46.935741+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:33.786643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018329830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6854,"num_read_kb":23202,"num_write":1277,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"189'594","reported_seq":3645,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935677+0000","last_change":"2024-09-13T09:10:27.288793+0000","last_active":"2024-09-13T09:10:46.935677+0000","last_peered":"2024-09-13T09:10:46.935677+0000","last_clean":"2024-09-13T09:10:46.935677+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:46.935677+0000","last_undegraded":"2024-09-13T09:10:46.935677+0000","last_fullsized":"2024-09-13T09:10:46.935677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099698400000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4866,"num_read_kb":16848,"num_write":688,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"188'590","reported_seq":3110,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936333+0000","last_change":"2024-09-13T09:10:27.326271+0000","last_active":"2024-09-13T09:10:46.936333+0000","last_peered":"2024-09-13T09:10:46.936333+0000","last_clean":"2024-09-13T09:10:46.936333+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:46.936333+0000","last_undegraded":"2024-09-13T09:10:46.936333+0000","last_fullsized":"2024-09-13T09:10:46.936333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074287699999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4001,"num_read_kb":22630,"num_write":547,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"188'428","reported_seq":3878,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424141+0000","last_change":"2024-09-13T09:10:27.294587+0000","last_active":"2024-09-13T09:10:46.424141+0000","last_peered":"2024-09-13T09:10:46.424141+0000","last_clean":"2024-09-13T09:10:46.424141+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:10:46.424141+0000","last_undegraded":"2024-09-13T09:10:46.424141+0000","last_fullsized":"2024-09-13T09:10:46.424141+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:11.164172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013662279999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3594,"num_read_kb":33920,"num_write":421,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"187'360","reported_seq":17058,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002334+0000","last_change":"2024-09-13T09:10:35.921897+0000","last_active":"2024-09-13T09:10:46.002334+0000","last_peered":"2024-09-13T09:10:46.002334+0000","last_clean":"2024-09-13T09:10:46.002334+0000","last_became_active":"2024-09-13T09:10:35.919376+0000","last_became_peered":"2024-09-13T09:10:35.919376+0000","last_unstale":"2024-09-13T09:10:46.002334+0000","last_undegraded":"2024-09-13T09:10:46.002334+0000","last_fullsized":"2024-09-13T09:10:46.002334+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:08.251141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088103200000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":16596,"num_read_kb":27933,"num_write":361,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"190'410","reported_seq":3137,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935350+0000","last_change":"2024-09-13T09:10:27.289763+0000","last_active":"2024-09-13T09:10:46.935350+0000","last_peered":"2024-09-13T09:10:46.935350+0000","last_clean":"2024-09-13T09:10:46.935350+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:46.935350+0000","last_undegraded":"2024-09-13T09:10:46.935350+0000","last_fullsized":"2024-09-13T09:10:46.935350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001690069,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3095,"num_read_kb":9820,"num_write":360,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"190'884","reported_seq":7372,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424756+0000","last_change":"2024-09-13T09:10:35.916276+0000","last_active":"2024-09-13T09:10:46.424756+0000","last_peered":"2024-09-13T09:10:46.424756+0000","last_clean":"2024-09-13T09:10:46.424756+0000","last_became_active":"2024-09-13T09:10:35.914456+0000","last_became_peered":"2024-09-13T09:10:35.914456+0000","last_unstale":"2024-09-13T09:10:46.424756+0000","last_undegraded":"2024-09-13T09:10:46.424756+0000","last_fullsized":"2024-09-13T09:10:46.424756+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:13.805979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001663692,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9292,"num_read_kb":23776,"num_write":1223,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"189'477","reported_seq":5178,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002164+0000","last_change":"2024-09-13T09:10:35.918190+0000","last_active":"2024-09-13T09:10:46.002164+0000","last_peered":"2024-09-13T09:10:46.002164+0000","last_clean":"2024-09-13T09:10:46.002164+0000","last_became_active":"2024-09-13T09:10:35.915413+0000","last_became_peered":"2024-09-13T09:10:35.915413+0000","last_unstale":"2024-09-13T09:10:46.002164+0000","last_undegraded":"2024-09-13T09:10:46.002164+0000","last_fullsized":"2024-09-13T09:10:46.002164+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:23.432147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001550994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5016,"num_read_kb":17262,"num_write":425,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"188'518","reported_seq":2907,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435248+0000","last_change":"2024-09-13T09:10:27.300360+0000","last_active":"2024-09-13T09:10:46.435248+0000","last_peered":"2024-09-13T09:10:46.435248+0000","last_clean":"2024-09-13T09:10:46.435248+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:10:46.435248+0000","last_undegraded":"2024-09-13T09:10:46.435248+0000","last_fullsized":"2024-09-13T09:10:46.435248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011980339999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2903,"num_read_kb":17169,"num_write":491,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"190'1187","reported_seq":4411,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935485+0000","last_change":"2024-09-13T09:10:27.289049+0000","last_active":"2024-09-13T09:10:46.935485+0000","last_peered":"2024-09-13T09:10:46.935485+0000","last_clean":"2024-09-13T09:10:46.935485+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:46.935485+0000","last_undegraded":"2024-09-13T09:10:46.935485+0000","last_fullsized":"2024-09-13T09:10:46.935485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001456372,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7816,"num_read_kb":12367,"num_write":1664,"num_write_kb":5387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"189'718","reported_seq":4615,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000641+0000","last_change":"2024-09-13T09:10:27.287499+0000","last_active":"2024-09-13T09:10:46.000641+0000","last_peered":"2024-09-13T09:10:46.000641+0000","last_clean":"2024-09-13T09:10:46.000641+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:46.000641+0000","last_undegraded":"2024-09-13T09:10:46.000641+0000","last_fullsized":"2024-09-13T09:10:46.000641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012075230000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5826,"num_read_kb":14063,"num_write":971,"num_write_kb":11379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"187'588","reported_seq":4153,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935575+0000","last_change":"2024-09-13T09:10:31.329333+0000","last_active":"2024-09-13T09:10:46.935575+0000","last_peered":"2024-09-13T09:10:46.935575+0000","last_clean":"2024-09-13T09:10:46.935575+0000","last_became_active":"2024-09-13T09:10:31.325692+0000","last_became_peered":"2024-09-13T09:10:31.325692+0000","last_unstale":"2024-09-13T09:10:46.935575+0000","last_undegraded":"2024-09-13T09:10:46.935575+0000","last_fullsized":"2024-09-13T09:10:46.935575+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:57.571689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015796709999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3375,"num_read_kb":13502,"num_write":606,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":336,"num_read":210850,"num_read_kb":690514,"num_write":25042,"num_write_kb":338108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3692,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":21420,"ondisk_log_size":21420,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080584,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":2472,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65584,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306256,"statfs":{"total":96636764160,"available":96569606144,"internally_reserved":0,"allocated":2523136,"data_stored":1220555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64616904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.22,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.875,"5min":1.361,"15min":1.361},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63500000000000001,"15min":0.64700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.1459999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.747}]},{"osd":1,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63600000000000001},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.88100000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":1.444,"15min":1.444},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1850000000000001,"5min":1.766,"15min":1.766},"last":0.88}]},{"osd":4,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2170000000000001,"5min":1.343,"15min":1.343},"last":1.137}]},{"osd":5,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.919,"15min":1.919},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.90400000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4019999999999999,"5min":1.867,"15min":1.867},"last":1.081},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.238}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.2789999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69676,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":2519040,"data_stored":1220536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.67300000000000004,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.226,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73099999999999998,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.024,"5min":1.353,"15min":1.353},"last":0.504}]},{"osd":1,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.626},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.2350000000000001,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.401,"5min":1.681,"15min":1.681},"last":0.437},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.3,"5min":1.401,"15min":1.401},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":1.0249999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.288,"5min":1.998,"15min":1.998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.391,"5min":1.651,"15min":1.651},"last":0.879}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4379999999999999,"5min":1.506,"15min":1.506},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0009999999999999,"15min":1.0449999999999999},"min":{"1min":0.48899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.571,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.071},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3500000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.089,"15min":1.125},"min":{"1min":0.621,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.417,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":0.83599999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56316,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315524,"statfs":{"total":96636764160,"available":96579096576,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.30199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.86399999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.628}]},{"osd":2,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.624,"15min":0.64400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.054},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.90500000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.72399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0900000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3480000000000001,"5min":1.407,"15min":1.407},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3280000000000001,"5min":1.415,"15min":1.415},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.335,"5min":1.448,"15min":1.448},"last":1.145}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94399999999999995,"15min":0.97899999999999998},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.268},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1539999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.228}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93100000000000005,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4510000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.111},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.05},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.49,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.369},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.3029999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33588,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338252,"statfs":{"total":96636764160,"available":96602370048,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.59599999999999997},"min":{"1min":0.317,"5min":0.245,"15min":0.245},"max":{"1min":0.98099999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.625},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.379}]},{"osd":2,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66000000000000003,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.79200000000000004,"15min":0.86699999999999999},"min":{"1min":0.38500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.149,"5min":3.855,"15min":3.855},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0309999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.432},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.82999999999999996,"15min":0.92600000000000005},"min":{"1min":0.40999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.341,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.58899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":0.93799999999999994},"min":{"1min":0.54800000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.218,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.63}]},{"osd":5,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.94099999999999995,"15min":0.97399999999999998},"min":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2949999999999999,"5min":2.532,"15min":2.532},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.0629999999999999,"15min":1.123},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.516,"5min":2.181,"15min":2.181},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.056,"15min":1.1240000000000001},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3700000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.018,"15min":1.073},"min":{"1min":0.63100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4910000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:08:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.528,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0840000000000001,"15min":1.1220000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542195,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37692,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334148,"statfs":{"total":96636764160,"available":96598167552,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.434,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1759999999999999,"5min":1.552,"15min":1.552},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.446,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.752}]},{"osd":1,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.79600000000000004},"min":{"1min":0.47799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2989999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":1.097},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.92000000000000004},"min":{"1min":0.53200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2669999999999999,"5min":2.706,"15min":2.706},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.93600000000000005,"15min":0.89700000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.649,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.143},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.58899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1439999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.93400000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.022},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.92200000000000004,"15min":0.96399999999999997},"min":{"1min":0.498,"5min":0.377,"15min":0.377},"max":{"1min":1.5489999999999999,"5min":1.7,"15min":1.7},"last":1.202}]},{"osd":5,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.046,"15min":1.0800000000000001},"min":{"1min":0.5,"5min":0.497,"15min":0.497},"max":{"1min":1.359,"5min":1.776,"15min":1.776},"last":1.266},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.04,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.6220000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.323}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.4139999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3520000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542194,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.121,"5min":2.649,"15min":2.649},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.202,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1819999999999999,"5min":1.3,"15min":1.3},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.88,"15min":0.83099999999999996},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.996},{"interface":"front","average":{"1min":1.042,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.03}]},{"osd":3,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.91600000000000004,"15min":0.86799999999999999},"min":{"1min":0.623,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.591,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.149},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.98999999999999999,"15min":0.98199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.90100000000000002,"15min":0.85799999999999998},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.008,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.95899999999999996,"15min":0.89000000000000001},"min":{"1min":0.54900000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.74,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.2450000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.98599999999999999,"15min":0.94699999999999995},"min":{"1min":0.44600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.042,"15min":1.0389999999999999},"min":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.21}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.284}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.032,"15min":1.1499999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1930000000000001,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":1.194}]},{"osd":1,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.83599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2490000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.335,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":1.0269999999999999,"15min":1.1319999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2809999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.1990000000000001},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3799999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":1.145},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.151,"5min":1.585,"15min":1.585},"last":1.034}]},{"osd":4,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.997,"15min":1.024},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4119999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.321},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.25,"15min":1.3799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.351,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.417}]},{"osd":6,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.97999999999999998,"15min":0.98399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.278,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.355,"5min":1.899,"15min":1.899},"last":1.518}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.5509999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:10:52.019 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T09:10:52.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T09:10:52.340 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T09:10:52.352 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T09:10:52.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:10:52.641 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:52.642 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:10:52.658 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-13T09:10:51.112377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":336,"num_read":210966,"num_read_kb":690614,"num_write":25203,"num_write_kb":340526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21540,"ondisk_log_size":21540,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406268,"kb_used_data":20928,"kb_used_omap":30,"kb_used_meta":385185,"kb_avail":754568452,"statfs":{"total":773094113280,"available":772678094848,"internally_reserved":0,"allocated":21430272,"data_stored":10980674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31401,"internal_metadata":394429783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.124050"},"pg_stats":[{"pgid":"3.1f","version":"190'569","reported_seq":5735,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001506+0000","last_change":"2024-09-13T09:10:27.358289+0000","last_active":"2024-09-13T09:10:46.001506+0000","last_peered":"2024-09-13T09:10:46.001506+0000","last_clean":"2024-09-13T09:10:46.001506+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:46.001506+0000","last_undegraded":"2024-09-13T09:10:46.001506+0000","last_fullsized":"2024-09-13T09:10:46.001506+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016218700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5971,"num_read_kb":20218,"num_write":677,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"189'1226","reported_seq":5143,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001090+0000","last_change":"2024-09-13T09:10:27.357623+0000","last_active":"2024-09-13T09:10:46.001090+0000","last_peered":"2024-09-13T09:10:46.001090+0000","last_clean":"2024-09-13T09:10:46.001090+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:10:46.001090+0000","last_undegraded":"2024-09-13T09:10:46.001090+0000","last_fullsized":"2024-09-13T09:10:46.001090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082279900000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9326,"num_read_kb":19231,"num_write":1571,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"188'1409","reported_seq":5692,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935670+0000","last_change":"2024-09-13T09:10:37.937627+0000","last_active":"2024-09-13T09:10:46.935670+0000","last_peered":"2024-09-13T09:10:46.935670+0000","last_clean":"2024-09-13T09:10:46.935670+0000","last_became_active":"2024-09-13T09:10:37.936189+0000","last_became_peered":"2024-09-13T09:10:37.936189+0000","last_unstale":"2024-09-13T09:10:46.935670+0000","last_undegraded":"2024-09-13T09:10:46.935670+0000","last_fullsized":"2024-09-13T09:10:46.935670+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:43.640432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013860960000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8733,"num_read_kb":34149,"num_write":2234,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"190'732","reported_seq":4385,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435424+0000","last_change":"2024-09-13T09:10:36.914556+0000","last_active":"2024-09-13T09:10:46.435424+0000","last_peered":"2024-09-13T09:10:46.435424+0000","last_clean":"2024-09-13T09:10:46.435424+0000","last_became_active":"2024-09-13T09:10:36.913240+0000","last_became_peered":"2024-09-13T09:10:36.913240+0000","last_unstale":"2024-09-13T09:10:46.435424+0000","last_undegraded":"2024-09-13T09:10:46.435424+0000","last_fullsized":"2024-09-13T09:10:46.435424+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:19.776639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015845119999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5745,"num_read_kb":35631,"num_write":972,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"188'730","reported_seq":22464,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936232+0000","last_change":"2024-09-13T09:10:36.919210+0000","last_active":"2024-09-13T09:10:46.936232+0000","last_peered":"2024-09-13T09:10:46.936232+0000","last_clean":"2024-09-13T09:10:46.936232+0000","last_became_active":"2024-09-13T09:10:36.914093+0000","last_became_peered":"2024-09-13T09:10:36.914093+0000","last_unstale":"2024-09-13T09:10:46.936232+0000","last_undegraded":"2024-09-13T09:10:46.936232+0000","last_fullsized":"2024-09-13T09:10:46.936232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:46.181581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023266060000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":23676,"num_read_kb":48180,"num_write":787,"num_write_kb":18573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"188'704","reported_seq":6536,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435442+0000","last_change":"2024-09-13T09:10:27.300037+0000","last_active":"2024-09-13T09:10:46.435442+0000","last_peered":"2024-09-13T09:10:46.435442+0000","last_clean":"2024-09-13T09:10:46.435442+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:10:46.435442+0000","last_undegraded":"2024-09-13T09:10:46.435442+0000","last_fullsized":"2024-09-13T09:10:46.435442+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093814799999999998,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":8455,"num_read_kb":25867,"num_write":848,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"188'387","reported_seq":2116,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935630+0000","last_change":"2024-09-13T09:10:27.328428+0000","last_active":"2024-09-13T09:10:46.935630+0000","last_peered":"2024-09-13T09:10:46.935630+0000","last_clean":"2024-09-13T09:10:46.935630+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:10:46.935630+0000","last_undegraded":"2024-09-13T09:10:46.935630+0000","last_fullsized":"2024-09-13T09:10:46.935630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340506,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1827,"num_read_kb":8140,"num_write":348,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"189'714","reported_seq":3577,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001432+0000","last_change":"2024-09-13T09:10:27.358518+0000","last_active":"2024-09-13T09:10:46.001432+0000","last_peered":"2024-09-13T09:10:46.001432+0000","last_clean":"2024-09-13T09:10:46.001432+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:10:46.001432+0000","last_undegraded":"2024-09-13T09:10:46.001432+0000","last_fullsized":"2024-09-13T09:10:46.001432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016018569999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4805,"num_read_kb":14792,"num_write":763,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"189'668","reported_seq":5966,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001614+0000","last_change":"2024-09-13T09:10:27.358098+0000","last_active":"2024-09-13T09:10:46.001614+0000","last_peered":"2024-09-13T09:10:46.001614+0000","last_clean":"2024-09-13T09:10:46.001614+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:10:46.001614+0000","last_undegraded":"2024-09-13T09:10:46.001614+0000","last_fullsized":"2024-09-13T09:10:46.001614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689800000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7350,"num_read_kb":31391,"num_write":638,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"189'718","reported_seq":2129,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424676+0000","last_change":"2024-09-13T09:10:27.295495+0000","last_active":"2024-09-13T09:10:46.424676+0000","last_peered":"2024-09-13T09:10:46.424676+0000","last_clean":"2024-09-13T09:10:46.424676+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:46.424676+0000","last_undegraded":"2024-09-13T09:10:46.424676+0000","last_fullsized":"2024-09-13T09:10:46.424676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020123239999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2936,"num_read_kb":14217,"num_write":967,"num_write_kb":9811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"189'734","reported_seq":4409,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002493+0000","last_change":"2024-09-13T09:10:36.923247+0000","last_active":"2024-09-13T09:10:46.002493+0000","last_peered":"2024-09-13T09:10:46.002493+0000","last_clean":"2024-09-13T09:10:46.002493+0000","last_became_active":"2024-09-13T09:10:36.918448+0000","last_became_peered":"2024-09-13T09:10:36.918448+0000","last_unstale":"2024-09-13T09:10:46.002493+0000","last_undegraded":"2024-09-13T09:10:46.002493+0000","last_fullsized":"2024-09-13T09:10:46.002493+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:36.185691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021014559999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5305,"num_read_kb":24795,"num_write":695,"num_write_kb":16167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"190'725","reported_seq":19435,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435181+0000","last_change":"2024-09-13T09:10:27.300467+0000","last_active":"2024-09-13T09:10:46.435181+0000","last_peered":"2024-09-13T09:10:46.435181+0000","last_clean":"2024-09-13T09:10:46.435181+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:10:46.435181+0000","last_undegraded":"2024-09-13T09:10:46.435181+0000","last_fullsized":"2024-09-13T09:10:46.435181+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012366219999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21128,"num_read_kb":34179,"num_write":635,"num_write_kb":10198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":221,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:10:33.349791+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:10:33.349249+0000","last_became_peered":"2024-09-13T09:10:33.349249+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:24.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"190'700","reported_seq":6003,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935995+0000","last_change":"2024-09-13T09:10:27.329077+0000","last_active":"2024-09-13T09:10:46.935995+0000","last_peered":"2024-09-13T09:10:46.935995+0000","last_clean":"2024-09-13T09:10:46.935995+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:10:46.935995+0000","last_undegraded":"2024-09-13T09:10:46.935995+0000","last_fullsized":"2024-09-13T09:10:46.935995+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:13.066567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028872860000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6169,"num_read_kb":21942,"num_write":789,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936143+0000","last_change":"2024-09-13T09:10:33.349800+0000","last_active":"2024-09-13T09:10:46.936143+0000","last_peered":"2024-09-13T09:10:46.936143+0000","last_clean":"2024-09-13T09:10:46.936143+0000","last_became_active":"2024-09-13T09:10:33.349192+0000","last_became_peered":"2024-09-13T09:10:33.349192+0000","last_unstale":"2024-09-13T09:10:46.936143+0000","last_undegraded":"2024-09-13T09:10:46.936143+0000","last_fullsized":"2024-09-13T09:10:46.936143+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:16.106956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"189'433","reported_seq":2543,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936105+0000","last_change":"2024-09-13T09:10:27.327294+0000","last_active":"2024-09-13T09:10:46.936105+0000","last_peered":"2024-09-13T09:10:46.936105+0000","last_clean":"2024-09-13T09:10:46.936105+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:10:46.936105+0000","last_undegraded":"2024-09-13T09:10:46.936105+0000","last_fullsized":"2024-09-13T09:10:46.936105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001256109,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2732,"num_read_kb":14647,"num_write":388,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"188'621","reported_seq":6246,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001146+0000","last_change":"2024-09-13T09:10:27.286574+0000","last_active":"2024-09-13T09:10:46.001146+0000","last_peered":"2024-09-13T09:10:46.001146+0000","last_clean":"2024-09-13T09:10:46.001146+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:10:46.001146+0000","last_undegraded":"2024-09-13T09:10:46.001146+0000","last_fullsized":"2024-09-13T09:10:46.001146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043649599999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6810,"num_read_kb":26019,"num_write":814,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001342+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:10:46.001342+0000","last_peered":"2024-09-13T09:10:46.001342+0000","last_clean":"2024-09-13T09:10:46.001342+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:10:46.001342+0000","last_undegraded":"2024-09-13T09:10:46.001342+0000","last_fullsized":"2024-09-13T09:10:46.001342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"189'717","reported_seq":3887,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424566+0000","last_change":"2024-09-13T09:10:27.294511+0000","last_active":"2024-09-13T09:10:46.424566+0000","last_peered":"2024-09-13T09:10:46.424566+0000","last_clean":"2024-09-13T09:10:46.424566+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:10:46.424566+0000","last_undegraded":"2024-09-13T09:10:46.424566+0000","last_fullsized":"2024-09-13T09:10:46.424566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014721929999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4939,"num_read_kb":19757,"num_write":904,"num_write_kb":10484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"190'667","reported_seq":3539,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002172+0000","last_change":"2024-09-13T09:10:36.919578+0000","last_active":"2024-09-13T09:10:46.002172+0000","last_peered":"2024-09-13T09:10:46.002172+0000","last_clean":"2024-09-13T09:10:46.002172+0000","last_became_active":"2024-09-13T09:10:36.917124+0000","last_became_peered":"2024-09-13T09:10:36.917124+0000","last_unstale":"2024-09-13T09:10:46.002172+0000","last_undegraded":"2024-09-13T09:10:46.002172+0000","last_fullsized":"2024-09-13T09:10:46.002172+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:11.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011757899999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3513,"num_read_kb":18416,"num_write":638,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":207,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935660+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:10:46.935660+0000","last_peered":"2024-09-13T09:10:46.935660+0000","last_clean":"2024-09-13T09:10:46.935660+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:10:46.935660+0000","last_undegraded":"2024-09-13T09:10:46.935660+0000","last_fullsized":"2024-09-13T09:10:46.935660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"189'504","reported_seq":3792,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001037+0000","last_change":"2024-09-13T09:10:27.357629+0000","last_active":"2024-09-13T09:10:46.001037+0000","last_peered":"2024-09-13T09:10:46.001037+0000","last_clean":"2024-09-13T09:10:46.001037+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:10:46.001037+0000","last_undegraded":"2024-09-13T09:10:46.001037+0000","last_fullsized":"2024-09-13T09:10:46.001037+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:27.395379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095688200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3585,"num_read_kb":19320,"num_write":505,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935744+0000","last_change":"2024-09-13T09:10:19.191965+0000","last_active":"2024-09-13T09:10:46.935744+0000","last_peered":"2024-09-13T09:10:46.935744+0000","last_clean":"2024-09-13T09:10:46.935744+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:46.935744+0000","last_undegraded":"2024-09-13T09:10:46.935744+0000","last_fullsized":"2024-09-13T09:10:46.935744+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:17.687266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"190'422","reported_seq":2354,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000648+0000","last_change":"2024-09-13T09:10:27.286775+0000","last_active":"2024-09-13T09:10:46.000648+0000","last_peered":"2024-09-13T09:10:46.000648+0000","last_clean":"2024-09-13T09:10:46.000648+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:10:46.000648+0000","last_undegraded":"2024-09-13T09:10:46.000648+0000","last_fullsized":"2024-09-13T09:10:46.000648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074152199999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2071,"num_read_kb":14898,"num_write":412,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935439+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:10:46.935439+0000","last_peered":"2024-09-13T09:10:46.935439+0000","last_clean":"2024-09-13T09:10:46.935439+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:10:46.935439+0000","last_undegraded":"2024-09-13T09:10:46.935439+0000","last_fullsized":"2024-09-13T09:10:46.935439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"189'405","reported_seq":2579,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936353+0000","last_change":"2024-09-13T09:10:27.326747+0000","last_active":"2024-09-13T09:10:46.936353+0000","last_peered":"2024-09-13T09:10:46.936353+0000","last_clean":"2024-09-13T09:10:46.936353+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:10:46.936353+0000","last_undegraded":"2024-09-13T09:10:46.936353+0000","last_fullsized":"2024-09-13T09:10:46.936353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037313,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2540,"num_read_kb":12233,"num_write":423,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935747+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:10:46.935747+0000","last_peered":"2024-09-13T09:10:46.935747+0000","last_clean":"2024-09-13T09:10:46.935747+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:10:46.935747+0000","last_undegraded":"2024-09-13T09:10:46.935747+0000","last_fullsized":"2024-09-13T09:10:46.935747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":335,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435002+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:10:46.435002+0000","last_peered":"2024-09-13T09:10:46.435002+0000","last_clean":"2024-09-13T09:10:46.435002+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:10:46.435002+0000","last_undegraded":"2024-09-13T09:10:46.435002+0000","last_fullsized":"2024-09-13T09:10:46.435002+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"190'881","reported_seq":6152,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935741+0000","last_change":"2024-09-13T09:10:27.327530+0000","last_active":"2024-09-13T09:10:46.935741+0000","last_peered":"2024-09-13T09:10:46.935741+0000","last_clean":"2024-09-13T09:10:46.935741+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:10:46.935741+0000","last_undegraded":"2024-09-13T09:10:46.935741+0000","last_fullsized":"2024-09-13T09:10:46.935741+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:33.786643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018329830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6854,"num_read_kb":23202,"num_write":1277,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"189'594","reported_seq":3645,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935677+0000","last_change":"2024-09-13T09:10:27.288793+0000","last_active":"2024-09-13T09:10:46.935677+0000","last_peered":"2024-09-13T09:10:46.935677+0000","last_clean":"2024-09-13T09:10:46.935677+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:46.935677+0000","last_undegraded":"2024-09-13T09:10:46.935677+0000","last_fullsized":"2024-09-13T09:10:46.935677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099698400000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4866,"num_read_kb":16848,"num_write":688,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"188'590","reported_seq":3110,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936333+0000","last_change":"2024-09-13T09:10:27.326271+0000","last_active":"2024-09-13T09:10:46.936333+0000","last_peered":"2024-09-13T09:10:46.936333+0000","last_clean":"2024-09-13T09:10:46.936333+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:46.936333+0000","last_undegraded":"2024-09-13T09:10:46.936333+0000","last_fullsized":"2024-09-13T09:10:46.936333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074287699999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4001,"num_read_kb":22630,"num_write":547,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"188'428","reported_seq":3878,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424141+0000","last_change":"2024-09-13T09:10:27.294587+0000","last_active":"2024-09-13T09:10:46.424141+0000","last_peered":"2024-09-13T09:10:46.424141+0000","last_clean":"2024-09-13T09:10:46.424141+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:10:46.424141+0000","last_undegraded":"2024-09-13T09:10:46.424141+0000","last_fullsized":"2024-09-13T09:10:46.424141+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:11.164172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013662279999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3594,"num_read_kb":33920,"num_write":421,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"187'360","reported_seq":17058,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002334+0000","last_change":"2024-09-13T09:10:35.921897+0000","last_active":"2024-09-13T09:10:46.002334+0000","last_peered":"2024-09-13T09:10:46.002334+0000","last_clean":"2024-09-13T09:10:46.002334+0000","last_became_active":"2024-09-13T09:10:35.919376+0000","last_became_peered":"2024-09-13T09:10:35.919376+0000","last_unstale":"2024-09-13T09:10:46.002334+0000","last_undegraded":"2024-09-13T09:10:46.002334+0000","last_fullsized":"2024-09-13T09:10:46.002334+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:08.251141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088103200000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":16596,"num_read_kb":27933,"num_write":361,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"190'410","reported_seq":3137,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935350+0000","last_change":"2024-09-13T09:10:27.289763+0000","last_active":"2024-09-13T09:10:46.935350+0000","last_peered":"2024-09-13T09:10:46.935350+0000","last_clean":"2024-09-13T09:10:46.935350+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:46.935350+0000","last_undegraded":"2024-09-13T09:10:46.935350+0000","last_fullsized":"2024-09-13T09:10:46.935350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001690069,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3095,"num_read_kb":9820,"num_write":360,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"190'884","reported_seq":7372,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424756+0000","last_change":"2024-09-13T09:10:35.916276+0000","last_active":"2024-09-13T09:10:46.424756+0000","last_peered":"2024-09-13T09:10:46.424756+0000","last_clean":"2024-09-13T09:10:46.424756+0000","last_became_active":"2024-09-13T09:10:35.914456+0000","last_became_peered":"2024-09-13T09:10:35.914456+0000","last_unstale":"2024-09-13T09:10:46.424756+0000","last_undegraded":"2024-09-13T09:10:46.424756+0000","last_fullsized":"2024-09-13T09:10:46.424756+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:13.805979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001663692,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9292,"num_read_kb":23776,"num_write":1223,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"189'477","reported_seq":5178,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002164+0000","last_change":"2024-09-13T09:10:35.918190+0000","last_active":"2024-09-13T09:10:46.002164+0000","last_peered":"2024-09-13T09:10:46.002164+0000","last_clean":"2024-09-13T09:10:46.002164+0000","last_became_active":"2024-09-13T09:10:35.915413+0000","last_became_peered":"2024-09-13T09:10:35.915413+0000","last_unstale":"2024-09-13T09:10:46.002164+0000","last_undegraded":"2024-09-13T09:10:46.002164+0000","last_fullsized":"2024-09-13T09:10:46.002164+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:23.432147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001550994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5016,"num_read_kb":17262,"num_write":425,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"188'518","reported_seq":2907,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435248+0000","last_change":"2024-09-13T09:10:27.300360+0000","last_active":"2024-09-13T09:10:46.435248+0000","last_peered":"2024-09-13T09:10:46.435248+0000","last_clean":"2024-09-13T09:10:46.435248+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:10:46.435248+0000","last_undegraded":"2024-09-13T09:10:46.435248+0000","last_fullsized":"2024-09-13T09:10:46.435248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011980339999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2903,"num_read_kb":17169,"num_write":491,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"190'1187","reported_seq":4411,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935485+0000","last_change":"2024-09-13T09:10:27.289049+0000","last_active":"2024-09-13T09:10:46.935485+0000","last_peered":"2024-09-13T09:10:46.935485+0000","last_clean":"2024-09-13T09:10:46.935485+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:46.935485+0000","last_undegraded":"2024-09-13T09:10:46.935485+0000","last_fullsized":"2024-09-13T09:10:46.935485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001456372,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7816,"num_read_kb":12367,"num_write":1664,"num_write_kb":5387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"189'718","reported_seq":4615,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000641+0000","last_change":"2024-09-13T09:10:27.287499+0000","last_active":"2024-09-13T09:10:46.000641+0000","last_peered":"2024-09-13T09:10:46.000641+0000","last_clean":"2024-09-13T09:10:46.000641+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:46.000641+0000","last_undegraded":"2024-09-13T09:10:46.000641+0000","last_fullsized":"2024-09-13T09:10:46.000641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012075230000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5826,"num_read_kb":14063,"num_write":971,"num_write_kb":11379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"187'588","reported_seq":4153,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935575+0000","last_change":"2024-09-13T09:10:31.329333+0000","last_active":"2024-09-13T09:10:46.935575+0000","last_peered":"2024-09-13T09:10:46.935575+0000","last_clean":"2024-09-13T09:10:46.935575+0000","last_became_active":"2024-09-13T09:10:31.325692+0000","last_became_peered":"2024-09-13T09:10:31.325692+0000","last_unstale":"2024-09-13T09:10:46.935575+0000","last_undegraded":"2024-09-13T09:10:46.935575+0000","last_fullsized":"2024-09-13T09:10:46.935575+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:57.571689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015796709999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3375,"num_read_kb":13502,"num_write":606,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":336,"num_read":210850,"num_read_kb":690514,"num_write":25042,"num_write_kb":338108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3692,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":21420,"ondisk_log_size":21420,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080584,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":2472,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65584,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306256,"statfs":{"total":96636764160,"available":96569606144,"internally_reserved":0,"allocated":2523136,"data_stored":1220555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64616904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.22,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.875,"5min":1.361,"15min":1.361},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63500000000000001,"15min":0.64700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.1459999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.747}]},{"osd":1,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63600000000000001},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.88100000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":1.444,"15min":1.444},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1850000000000001,"5min":1.766,"15min":1.766},"last":0.88}]},{"osd":4,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2170000000000001,"5min":1.343,"15min":1.343},"last":1.137}]},{"osd":5,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.919,"15min":1.919},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.90400000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4019999999999999,"5min":1.867,"15min":1.867},"last":1.081},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.238}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.2789999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69676,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":2519040,"data_stored":1220536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.67300000000000004,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.226,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73099999999999998,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.024,"5min":1.353,"15min":1.353},"last":0.504}]},{"osd":1,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.626},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.2350000000000001,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.401,"5min":1.681,"15min":1.681},"last":0.437},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.3,"5min":1.401,"15min":1.401},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":1.0249999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.288,"5min":1.998,"15min":1.998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.391,"5min":1.651,"15min":1.651},"last":0.879}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4379999999999999,"5min":1.506,"15min":1.506},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0009999999999999,"15min":1.0449999999999999},"min":{"1min":0.48899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.571,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.071},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3500000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.089,"15min":1.125},"min":{"1min":0.621,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.417,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":0.83599999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56316,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315524,"statfs":{"total":96636764160,"available":96579096576,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.30199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.86399999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.628}]},{"osd":2,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.624,"15min":0.64400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.054},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.90500000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.72399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0900000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3480000000000001,"5min":1.407,"15min":1.407},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3280000000000001,"5min":1.415,"15min":1.415},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.335,"5min":1.448,"15min":1.448},"last":1.145}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94399999999999995,"15min":0.97899999999999998},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.268},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1539999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.228}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93100000000000005,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4510000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.111},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.05},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.49,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.369},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.3029999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33588,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338252,"statfs":{"total":96636764160,"available":96602370048,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.59599999999999997},"min":{"1min":0.317,"5min":0.245,"15min":0.245},"max":{"1min":0.98099999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.625},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.379}]},{"osd":2,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66000000000000003,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.79200000000000004,"15min":0.86699999999999999},"min":{"1min":0.38500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.149,"5min":3.855,"15min":3.855},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0309999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.432},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.82999999999999996,"15min":0.92600000000000005},"min":{"1min":0.40999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.341,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.58899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":0.93799999999999994},"min":{"1min":0.54800000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.218,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.63}]},{"osd":5,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.94099999999999995,"15min":0.97399999999999998},"min":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2949999999999999,"5min":2.532,"15min":2.532},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.0629999999999999,"15min":1.123},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.516,"5min":2.181,"15min":2.181},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.056,"15min":1.1240000000000001},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3700000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.018,"15min":1.073},"min":{"1min":0.63100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4910000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:08:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.528,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0840000000000001,"15min":1.1220000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542195,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37692,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334148,"statfs":{"total":96636764160,"available":96598167552,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.434,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1759999999999999,"5min":1.552,"15min":1.552},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.446,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.752}]},{"osd":1,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.79600000000000004},"min":{"1min":0.47799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2989999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":1.097},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.92000000000000004},"min":{"1min":0.53200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2669999999999999,"5min":2.706,"15min":2.706},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.93600000000000005,"15min":0.89700000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.649,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.143},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.58899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1439999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.93400000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.022},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.92200000000000004,"15min":0.96399999999999997},"min":{"1min":0.498,"5min":0.377,"15min":0.377},"max":{"1min":1.5489999999999999,"5min":1.7,"15min":1.7},"last":1.202}]},{"osd":5,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.046,"15min":1.0800000000000001},"min":{"1min":0.5,"5min":0.497,"15min":0.497},"max":{"1min":1.359,"5min":1.776,"15min":1.776},"last":1.266},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.04,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.6220000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.323}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.4139999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3520000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542194,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.121,"5min":2.649,"15min":2.649},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.202,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1819999999999999,"5min":1.3,"15min":1.3},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.88,"15min":0.83099999999999996},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.996},{"interface":"front","average":{"1min":1.042,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.03}]},{"osd":3,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.91600000000000004,"15min":0.86799999999999999},"min":{"1min":0.623,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.591,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.149},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.98999999999999999,"15min":0.98199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.90100000000000002,"15min":0.85799999999999998},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.008,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.95899999999999996,"15min":0.89000000000000001},"min":{"1min":0.54900000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.74,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.2450000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.98599999999999999,"15min":0.94699999999999995},"min":{"1min":0.44600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.042,"15min":1.0389999999999999},"min":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.21}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.284}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.032,"15min":1.1499999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1930000000000001,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":1.194}]},{"osd":1,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.83599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2490000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.335,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":1.0269999999999999,"15min":1.1319999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2809999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.1990000000000001},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3799999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":1.145},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.151,"5min":1.585,"15min":1.585},"last":1.034}]},{"osd":4,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.997,"15min":1.024},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4119999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.321},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.25,"15min":1.3799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.351,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.417}]},{"osd":6,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.97999999999999998,"15min":0.98399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.278,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.355,"5min":1.899,"15min":1.899},"last":1.518}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.5509999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:10:52.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:10:52.960 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:52.961 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:10:52.977 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-13T09:10:51.112377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":336,"num_read":210966,"num_read_kb":690614,"num_write":25203,"num_write_kb":340526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21540,"ondisk_log_size":21540,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406268,"kb_used_data":20928,"kb_used_omap":30,"kb_used_meta":385185,"kb_avail":754568452,"statfs":{"total":773094113280,"available":772678094848,"internally_reserved":0,"allocated":21430272,"data_stored":10980674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31401,"internal_metadata":394429783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.124050"},"pg_stats":[{"pgid":"3.1f","version":"190'569","reported_seq":5735,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001506+0000","last_change":"2024-09-13T09:10:27.358289+0000","last_active":"2024-09-13T09:10:46.001506+0000","last_peered":"2024-09-13T09:10:46.001506+0000","last_clean":"2024-09-13T09:10:46.001506+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:46.001506+0000","last_undegraded":"2024-09-13T09:10:46.001506+0000","last_fullsized":"2024-09-13T09:10:46.001506+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016218700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5971,"num_read_kb":20218,"num_write":677,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"189'1226","reported_seq":5143,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001090+0000","last_change":"2024-09-13T09:10:27.357623+0000","last_active":"2024-09-13T09:10:46.001090+0000","last_peered":"2024-09-13T09:10:46.001090+0000","last_clean":"2024-09-13T09:10:46.001090+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:10:46.001090+0000","last_undegraded":"2024-09-13T09:10:46.001090+0000","last_fullsized":"2024-09-13T09:10:46.001090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082279900000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9326,"num_read_kb":19231,"num_write":1571,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"188'1409","reported_seq":5692,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935670+0000","last_change":"2024-09-13T09:10:37.937627+0000","last_active":"2024-09-13T09:10:46.935670+0000","last_peered":"2024-09-13T09:10:46.935670+0000","last_clean":"2024-09-13T09:10:46.935670+0000","last_became_active":"2024-09-13T09:10:37.936189+0000","last_became_peered":"2024-09-13T09:10:37.936189+0000","last_unstale":"2024-09-13T09:10:46.935670+0000","last_undegraded":"2024-09-13T09:10:46.935670+0000","last_fullsized":"2024-09-13T09:10:46.935670+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:43.640432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013860960000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8733,"num_read_kb":34149,"num_write":2234,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"190'732","reported_seq":4385,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435424+0000","last_change":"2024-09-13T09:10:36.914556+0000","last_active":"2024-09-13T09:10:46.435424+0000","last_peered":"2024-09-13T09:10:46.435424+0000","last_clean":"2024-09-13T09:10:46.435424+0000","last_became_active":"2024-09-13T09:10:36.913240+0000","last_became_peered":"2024-09-13T09:10:36.913240+0000","last_unstale":"2024-09-13T09:10:46.435424+0000","last_undegraded":"2024-09-13T09:10:46.435424+0000","last_fullsized":"2024-09-13T09:10:46.435424+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:19.776639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015845119999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5745,"num_read_kb":35631,"num_write":972,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"188'730","reported_seq":22464,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936232+0000","last_change":"2024-09-13T09:10:36.919210+0000","last_active":"2024-09-13T09:10:46.936232+0000","last_peered":"2024-09-13T09:10:46.936232+0000","last_clean":"2024-09-13T09:10:46.936232+0000","last_became_active":"2024-09-13T09:10:36.914093+0000","last_became_peered":"2024-09-13T09:10:36.914093+0000","last_unstale":"2024-09-13T09:10:46.936232+0000","last_undegraded":"2024-09-13T09:10:46.936232+0000","last_fullsized":"2024-09-13T09:10:46.936232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:46.181581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023266060000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":23676,"num_read_kb":48180,"num_write":787,"num_write_kb":18573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"188'704","reported_seq":6536,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435442+0000","last_change":"2024-09-13T09:10:27.300037+0000","last_active":"2024-09-13T09:10:46.435442+0000","last_peered":"2024-09-13T09:10:46.435442+0000","last_clean":"2024-09-13T09:10:46.435442+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:10:46.435442+0000","last_undegraded":"2024-09-13T09:10:46.435442+0000","last_fullsized":"2024-09-13T09:10:46.435442+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093814799999999998,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":8455,"num_read_kb":25867,"num_write":848,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"188'387","reported_seq":2116,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935630+0000","last_change":"2024-09-13T09:10:27.328428+0000","last_active":"2024-09-13T09:10:46.935630+0000","last_peered":"2024-09-13T09:10:46.935630+0000","last_clean":"2024-09-13T09:10:46.935630+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:10:46.935630+0000","last_undegraded":"2024-09-13T09:10:46.935630+0000","last_fullsized":"2024-09-13T09:10:46.935630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340506,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1827,"num_read_kb":8140,"num_write":348,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"189'714","reported_seq":3577,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001432+0000","last_change":"2024-09-13T09:10:27.358518+0000","last_active":"2024-09-13T09:10:46.001432+0000","last_peered":"2024-09-13T09:10:46.001432+0000","last_clean":"2024-09-13T09:10:46.001432+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:10:46.001432+0000","last_undegraded":"2024-09-13T09:10:46.001432+0000","last_fullsized":"2024-09-13T09:10:46.001432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016018569999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4805,"num_read_kb":14792,"num_write":763,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"189'668","reported_seq":5966,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001614+0000","last_change":"2024-09-13T09:10:27.358098+0000","last_active":"2024-09-13T09:10:46.001614+0000","last_peered":"2024-09-13T09:10:46.001614+0000","last_clean":"2024-09-13T09:10:46.001614+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:10:46.001614+0000","last_undegraded":"2024-09-13T09:10:46.001614+0000","last_fullsized":"2024-09-13T09:10:46.001614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689800000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7350,"num_read_kb":31391,"num_write":638,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"189'718","reported_seq":2129,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424676+0000","last_change":"2024-09-13T09:10:27.295495+0000","last_active":"2024-09-13T09:10:46.424676+0000","last_peered":"2024-09-13T09:10:46.424676+0000","last_clean":"2024-09-13T09:10:46.424676+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:46.424676+0000","last_undegraded":"2024-09-13T09:10:46.424676+0000","last_fullsized":"2024-09-13T09:10:46.424676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020123239999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2936,"num_read_kb":14217,"num_write":967,"num_write_kb":9811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"189'734","reported_seq":4409,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002493+0000","last_change":"2024-09-13T09:10:36.923247+0000","last_active":"2024-09-13T09:10:46.002493+0000","last_peered":"2024-09-13T09:10:46.002493+0000","last_clean":"2024-09-13T09:10:46.002493+0000","last_became_active":"2024-09-13T09:10:36.918448+0000","last_became_peered":"2024-09-13T09:10:36.918448+0000","last_unstale":"2024-09-13T09:10:46.002493+0000","last_undegraded":"2024-09-13T09:10:46.002493+0000","last_fullsized":"2024-09-13T09:10:46.002493+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:36.185691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021014559999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5305,"num_read_kb":24795,"num_write":695,"num_write_kb":16167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"190'725","reported_seq":19435,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435181+0000","last_change":"2024-09-13T09:10:27.300467+0000","last_active":"2024-09-13T09:10:46.435181+0000","last_peered":"2024-09-13T09:10:46.435181+0000","last_clean":"2024-09-13T09:10:46.435181+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:10:46.435181+0000","last_undegraded":"2024-09-13T09:10:46.435181+0000","last_fullsized":"2024-09-13T09:10:46.435181+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012366219999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21128,"num_read_kb":34179,"num_write":635,"num_write_kb":10198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":221,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:10:33.349791+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:10:33.349249+0000","last_became_peered":"2024-09-13T09:10:33.349249+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:24.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"190'700","reported_seq":6003,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935995+0000","last_change":"2024-09-13T09:10:27.329077+0000","last_active":"2024-09-13T09:10:46.935995+0000","last_peered":"2024-09-13T09:10:46.935995+0000","last_clean":"2024-09-13T09:10:46.935995+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:10:46.935995+0000","last_undegraded":"2024-09-13T09:10:46.935995+0000","last_fullsized":"2024-09-13T09:10:46.935995+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:13.066567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028872860000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6169,"num_read_kb":21942,"num_write":789,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936143+0000","last_change":"2024-09-13T09:10:33.349800+0000","last_active":"2024-09-13T09:10:46.936143+0000","last_peered":"2024-09-13T09:10:46.936143+0000","last_clean":"2024-09-13T09:10:46.936143+0000","last_became_active":"2024-09-13T09:10:33.349192+0000","last_became_peered":"2024-09-13T09:10:33.349192+0000","last_unstale":"2024-09-13T09:10:46.936143+0000","last_undegraded":"2024-09-13T09:10:46.936143+0000","last_fullsized":"2024-09-13T09:10:46.936143+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:16.106956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"189'433","reported_seq":2543,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936105+0000","last_change":"2024-09-13T09:10:27.327294+0000","last_active":"2024-09-13T09:10:46.936105+0000","last_peered":"2024-09-13T09:10:46.936105+0000","last_clean":"2024-09-13T09:10:46.936105+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:10:46.936105+0000","last_undegraded":"2024-09-13T09:10:46.936105+0000","last_fullsized":"2024-09-13T09:10:46.936105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001256109,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2732,"num_read_kb":14647,"num_write":388,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"188'621","reported_seq":6246,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001146+0000","last_change":"2024-09-13T09:10:27.286574+0000","last_active":"2024-09-13T09:10:46.001146+0000","last_peered":"2024-09-13T09:10:46.001146+0000","last_clean":"2024-09-13T09:10:46.001146+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:10:46.001146+0000","last_undegraded":"2024-09-13T09:10:46.001146+0000","last_fullsized":"2024-09-13T09:10:46.001146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043649599999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6810,"num_read_kb":26019,"num_write":814,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001342+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:10:46.001342+0000","last_peered":"2024-09-13T09:10:46.001342+0000","last_clean":"2024-09-13T09:10:46.001342+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:10:46.001342+0000","last_undegraded":"2024-09-13T09:10:46.001342+0000","last_fullsized":"2024-09-13T09:10:46.001342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"189'717","reported_seq":3887,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424566+0000","last_change":"2024-09-13T09:10:27.294511+0000","last_active":"2024-09-13T09:10:46.424566+0000","last_peered":"2024-09-13T09:10:46.424566+0000","last_clean":"2024-09-13T09:10:46.424566+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:10:46.424566+0000","last_undegraded":"2024-09-13T09:10:46.424566+0000","last_fullsized":"2024-09-13T09:10:46.424566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014721929999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4939,"num_read_kb":19757,"num_write":904,"num_write_kb":10484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"190'667","reported_seq":3539,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002172+0000","last_change":"2024-09-13T09:10:36.919578+0000","last_active":"2024-09-13T09:10:46.002172+0000","last_peered":"2024-09-13T09:10:46.002172+0000","last_clean":"2024-09-13T09:10:46.002172+0000","last_became_active":"2024-09-13T09:10:36.917124+0000","last_became_peered":"2024-09-13T09:10:36.917124+0000","last_unstale":"2024-09-13T09:10:46.002172+0000","last_undegraded":"2024-09-13T09:10:46.002172+0000","last_fullsized":"2024-09-13T09:10:46.002172+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:11.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011757899999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3513,"num_read_kb":18416,"num_write":638,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":207,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935660+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:10:46.935660+0000","last_peered":"2024-09-13T09:10:46.935660+0000","last_clean":"2024-09-13T09:10:46.935660+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:10:46.935660+0000","last_undegraded":"2024-09-13T09:10:46.935660+0000","last_fullsized":"2024-09-13T09:10:46.935660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"189'504","reported_seq":3792,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001037+0000","last_change":"2024-09-13T09:10:27.357629+0000","last_active":"2024-09-13T09:10:46.001037+0000","last_peered":"2024-09-13T09:10:46.001037+0000","last_clean":"2024-09-13T09:10:46.001037+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:10:46.001037+0000","last_undegraded":"2024-09-13T09:10:46.001037+0000","last_fullsized":"2024-09-13T09:10:46.001037+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:27.395379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095688200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3585,"num_read_kb":19320,"num_write":505,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935744+0000","last_change":"2024-09-13T09:10:19.191965+0000","last_active":"2024-09-13T09:10:46.935744+0000","last_peered":"2024-09-13T09:10:46.935744+0000","last_clean":"2024-09-13T09:10:46.935744+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:46.935744+0000","last_undegraded":"2024-09-13T09:10:46.935744+0000","last_fullsized":"2024-09-13T09:10:46.935744+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:17.687266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"190'422","reported_seq":2354,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000648+0000","last_change":"2024-09-13T09:10:27.286775+0000","last_active":"2024-09-13T09:10:46.000648+0000","last_peered":"2024-09-13T09:10:46.000648+0000","last_clean":"2024-09-13T09:10:46.000648+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:10:46.000648+0000","last_undegraded":"2024-09-13T09:10:46.000648+0000","last_fullsized":"2024-09-13T09:10:46.000648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074152199999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2071,"num_read_kb":14898,"num_write":412,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935439+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:10:46.935439+0000","last_peered":"2024-09-13T09:10:46.935439+0000","last_clean":"2024-09-13T09:10:46.935439+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:10:46.935439+0000","last_undegraded":"2024-09-13T09:10:46.935439+0000","last_fullsized":"2024-09-13T09:10:46.935439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"189'405","reported_seq":2579,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936353+0000","last_change":"2024-09-13T09:10:27.326747+0000","last_active":"2024-09-13T09:10:46.936353+0000","last_peered":"2024-09-13T09:10:46.936353+0000","last_clean":"2024-09-13T09:10:46.936353+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:10:46.936353+0000","last_undegraded":"2024-09-13T09:10:46.936353+0000","last_fullsized":"2024-09-13T09:10:46.936353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037313,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2540,"num_read_kb":12233,"num_write":423,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935747+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:10:46.935747+0000","last_peered":"2024-09-13T09:10:46.935747+0000","last_clean":"2024-09-13T09:10:46.935747+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:10:46.935747+0000","last_undegraded":"2024-09-13T09:10:46.935747+0000","last_fullsized":"2024-09-13T09:10:46.935747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":335,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435002+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:10:46.435002+0000","last_peered":"2024-09-13T09:10:46.435002+0000","last_clean":"2024-09-13T09:10:46.435002+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:10:46.435002+0000","last_undegraded":"2024-09-13T09:10:46.435002+0000","last_fullsized":"2024-09-13T09:10:46.435002+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"190'881","reported_seq":6152,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935741+0000","last_change":"2024-09-13T09:10:27.327530+0000","last_active":"2024-09-13T09:10:46.935741+0000","last_peered":"2024-09-13T09:10:46.935741+0000","last_clean":"2024-09-13T09:10:46.935741+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:10:46.935741+0000","last_undegraded":"2024-09-13T09:10:46.935741+0000","last_fullsized":"2024-09-13T09:10:46.935741+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:33.786643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018329830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6854,"num_read_kb":23202,"num_write":1277,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"189'594","reported_seq":3645,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935677+0000","last_change":"2024-09-13T09:10:27.288793+0000","last_active":"2024-09-13T09:10:46.935677+0000","last_peered":"2024-09-13T09:10:46.935677+0000","last_clean":"2024-09-13T09:10:46.935677+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:46.935677+0000","last_undegraded":"2024-09-13T09:10:46.935677+0000","last_fullsized":"2024-09-13T09:10:46.935677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099698400000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4866,"num_read_kb":16848,"num_write":688,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"188'590","reported_seq":3110,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936333+0000","last_change":"2024-09-13T09:10:27.326271+0000","last_active":"2024-09-13T09:10:46.936333+0000","last_peered":"2024-09-13T09:10:46.936333+0000","last_clean":"2024-09-13T09:10:46.936333+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:46.936333+0000","last_undegraded":"2024-09-13T09:10:46.936333+0000","last_fullsized":"2024-09-13T09:10:46.936333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074287699999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4001,"num_read_kb":22630,"num_write":547,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"188'428","reported_seq":3878,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424141+0000","last_change":"2024-09-13T09:10:27.294587+0000","last_active":"2024-09-13T09:10:46.424141+0000","last_peered":"2024-09-13T09:10:46.424141+0000","last_clean":"2024-09-13T09:10:46.424141+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:10:46.424141+0000","last_undegraded":"2024-09-13T09:10:46.424141+0000","last_fullsized":"2024-09-13T09:10:46.424141+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:11.164172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013662279999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3594,"num_read_kb":33920,"num_write":421,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"187'360","reported_seq":17058,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002334+0000","last_change":"2024-09-13T09:10:35.921897+0000","last_active":"2024-09-13T09:10:46.002334+0000","last_peered":"2024-09-13T09:10:46.002334+0000","last_clean":"2024-09-13T09:10:46.002334+0000","last_became_active":"2024-09-13T09:10:35.919376+0000","last_became_peered":"2024-09-13T09:10:35.919376+0000","last_unstale":"2024-09-13T09:10:46.002334+0000","last_undegraded":"2024-09-13T09:10:46.002334+0000","last_fullsized":"2024-09-13T09:10:46.002334+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:08.251141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088103200000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":16596,"num_read_kb":27933,"num_write":361,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"190'410","reported_seq":3137,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935350+0000","last_change":"2024-09-13T09:10:27.289763+0000","last_active":"2024-09-13T09:10:46.935350+0000","last_peered":"2024-09-13T09:10:46.935350+0000","last_clean":"2024-09-13T09:10:46.935350+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:46.935350+0000","last_undegraded":"2024-09-13T09:10:46.935350+0000","last_fullsized":"2024-09-13T09:10:46.935350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001690069,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3095,"num_read_kb":9820,"num_write":360,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"190'884","reported_seq":7372,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424756+0000","last_change":"2024-09-13T09:10:35.916276+0000","last_active":"2024-09-13T09:10:46.424756+0000","last_peered":"2024-09-13T09:10:46.424756+0000","last_clean":"2024-09-13T09:10:46.424756+0000","last_became_active":"2024-09-13T09:10:35.914456+0000","last_became_peered":"2024-09-13T09:10:35.914456+0000","last_unstale":"2024-09-13T09:10:46.424756+0000","last_undegraded":"2024-09-13T09:10:46.424756+0000","last_fullsized":"2024-09-13T09:10:46.424756+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:13.805979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001663692,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9292,"num_read_kb":23776,"num_write":1223,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"189'477","reported_seq":5178,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002164+0000","last_change":"2024-09-13T09:10:35.918190+0000","last_active":"2024-09-13T09:10:46.002164+0000","last_peered":"2024-09-13T09:10:46.002164+0000","last_clean":"2024-09-13T09:10:46.002164+0000","last_became_active":"2024-09-13T09:10:35.915413+0000","last_became_peered":"2024-09-13T09:10:35.915413+0000","last_unstale":"2024-09-13T09:10:46.002164+0000","last_undegraded":"2024-09-13T09:10:46.002164+0000","last_fullsized":"2024-09-13T09:10:46.002164+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:23.432147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001550994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5016,"num_read_kb":17262,"num_write":425,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"188'518","reported_seq":2907,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435248+0000","last_change":"2024-09-13T09:10:27.300360+0000","last_active":"2024-09-13T09:10:46.435248+0000","last_peered":"2024-09-13T09:10:46.435248+0000","last_clean":"2024-09-13T09:10:46.435248+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:10:46.435248+0000","last_undegraded":"2024-09-13T09:10:46.435248+0000","last_fullsized":"2024-09-13T09:10:46.435248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011980339999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2903,"num_read_kb":17169,"num_write":491,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"190'1187","reported_seq":4411,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935485+0000","last_change":"2024-09-13T09:10:27.289049+0000","last_active":"2024-09-13T09:10:46.935485+0000","last_peered":"2024-09-13T09:10:46.935485+0000","last_clean":"2024-09-13T09:10:46.935485+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:46.935485+0000","last_undegraded":"2024-09-13T09:10:46.935485+0000","last_fullsized":"2024-09-13T09:10:46.935485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001456372,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7816,"num_read_kb":12367,"num_write":1664,"num_write_kb":5387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"189'718","reported_seq":4615,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000641+0000","last_change":"2024-09-13T09:10:27.287499+0000","last_active":"2024-09-13T09:10:46.000641+0000","last_peered":"2024-09-13T09:10:46.000641+0000","last_clean":"2024-09-13T09:10:46.000641+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:46.000641+0000","last_undegraded":"2024-09-13T09:10:46.000641+0000","last_fullsized":"2024-09-13T09:10:46.000641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012075230000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5826,"num_read_kb":14063,"num_write":971,"num_write_kb":11379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"187'588","reported_seq":4153,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935575+0000","last_change":"2024-09-13T09:10:31.329333+0000","last_active":"2024-09-13T09:10:46.935575+0000","last_peered":"2024-09-13T09:10:46.935575+0000","last_clean":"2024-09-13T09:10:46.935575+0000","last_became_active":"2024-09-13T09:10:31.325692+0000","last_became_peered":"2024-09-13T09:10:31.325692+0000","last_unstale":"2024-09-13T09:10:46.935575+0000","last_undegraded":"2024-09-13T09:10:46.935575+0000","last_fullsized":"2024-09-13T09:10:46.935575+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:57.571689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015796709999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3375,"num_read_kb":13502,"num_write":606,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":336,"num_read":210850,"num_read_kb":690514,"num_write":25042,"num_write_kb":338108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3692,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":21420,"ondisk_log_size":21420,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080584,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":2472,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65584,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306256,"statfs":{"total":96636764160,"available":96569606144,"internally_reserved":0,"allocated":2523136,"data_stored":1220555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64616904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.22,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.875,"5min":1.361,"15min":1.361},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63500000000000001,"15min":0.64700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.1459999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.747}]},{"osd":1,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63600000000000001},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.88100000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":1.444,"15min":1.444},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1850000000000001,"5min":1.766,"15min":1.766},"last":0.88}]},{"osd":4,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2170000000000001,"5min":1.343,"15min":1.343},"last":1.137}]},{"osd":5,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.919,"15min":1.919},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":1.0349999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.90400000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4019999999999999,"5min":1.867,"15min":1.867},"last":1.081},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.238}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.2789999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69676,"kb_used_data":2460,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":2519040,"data_stored":1220536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.67300000000000004,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.226,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73099999999999998,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.024,"5min":1.353,"15min":1.353},"last":0.504}]},{"osd":1,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.626},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.2350000000000001,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.401,"5min":1.681,"15min":1.681},"last":0.437},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.3,"5min":1.401,"15min":1.401},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":1.0249999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.288,"5min":1.998,"15min":1.998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.391,"5min":1.651,"15min":1.651},"last":0.879}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4379999999999999,"5min":1.506,"15min":1.506},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0009999999999999,"15min":1.0449999999999999},"min":{"1min":0.48899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.571,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.071},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3500000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.089,"15min":1.125},"min":{"1min":0.621,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.417,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.93899999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":0.83599999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56316,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315524,"statfs":{"total":96636764160,"available":96579096576,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.30199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.86399999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.628}]},{"osd":2,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.624,"15min":0.64400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.054},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.90500000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.72399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0900000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3480000000000001,"5min":1.407,"15min":1.407},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3280000000000001,"5min":1.415,"15min":1.415},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.335,"5min":1.448,"15min":1.448},"last":1.145}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94399999999999995,"15min":0.97899999999999998},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.268},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1539999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.228}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93100000000000005,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4510000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.111},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.05},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.49,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.369},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.3029999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33588,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338252,"statfs":{"total":96636764160,"available":96602370048,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.59599999999999997},"min":{"1min":0.317,"5min":0.245,"15min":0.245},"max":{"1min":0.98099999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.625},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.379}]},{"osd":2,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66000000000000003,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.79200000000000004,"15min":0.86699999999999999},"min":{"1min":0.38500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.149,"5min":3.855,"15min":3.855},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0309999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.432},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.82999999999999996,"15min":0.92600000000000005},"min":{"1min":0.40999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.341,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.58899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":0.93799999999999994},"min":{"1min":0.54800000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.218,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.63}]},{"osd":5,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.94099999999999995,"15min":0.97399999999999998},"min":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2949999999999999,"5min":2.532,"15min":2.532},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.997,"5min":1.0629999999999999,"15min":1.123},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.516,"5min":2.181,"15min":2.181},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.056,"15min":1.1240000000000001},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3700000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.018,"15min":1.073},"min":{"1min":0.63100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4910000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Sep 13 09:08:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.528,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0840000000000001,"15min":1.1220000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542195,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37692,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334148,"statfs":{"total":96636764160,"available":96598167552,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.434,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1759999999999999,"5min":1.552,"15min":1.552},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.446,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.752}]},{"osd":1,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.79600000000000004},"min":{"1min":0.47799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2989999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":1.097},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.92000000000000004},"min":{"1min":0.53200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2669999999999999,"5min":2.706,"15min":2.706},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.93600000000000005,"15min":0.89700000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.649,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.143},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.58899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1439999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.93400000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.022},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.92200000000000004,"15min":0.96399999999999997},"min":{"1min":0.498,"5min":0.377,"15min":0.377},"max":{"1min":1.5489999999999999,"5min":1.7,"15min":1.7},"last":1.202}]},{"osd":5,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.046,"15min":1.0800000000000001},"min":{"1min":0.5,"5min":0.497,"15min":0.497},"max":{"1min":1.359,"5min":1.776,"15min":1.776},"last":1.266},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.04,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.6220000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.323}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.4139999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3520000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542194,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.121,"5min":2.649,"15min":2.649},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.202,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1819999999999999,"5min":1.3,"15min":1.3},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.88,"15min":0.83099999999999996},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.996},{"interface":"front","average":{"1min":1.042,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.03}]},{"osd":3,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.91600000000000004,"15min":0.86799999999999999},"min":{"1min":0.623,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.591,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.149},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.98999999999999999,"15min":0.98199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.90100000000000002,"15min":0.85799999999999998},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.008,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.95899999999999996,"15min":0.89000000000000001},"min":{"1min":0.54900000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.74,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.2450000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.98599999999999999,"15min":0.94699999999999995},"min":{"1min":0.44600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.042,"15min":1.0389999999999999},"min":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.21}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.284}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.032,"15min":1.1499999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1930000000000001,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":1.194}]},{"osd":1,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.83599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2490000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.335,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":1.0269999999999999,"15min":1.1319999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2809999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.1990000000000001},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3799999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":1.145},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.151,"5min":1.585,"15min":1.585},"last":1.034}]},{"osd":4,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.997,"15min":1.024},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4119999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.321},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.25,"15min":1.3799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.351,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.417}]},{"osd":6,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.97999999999999998,"15min":0.98399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.278,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.355,"5min":1.899,"15min":1.899},"last":1.518}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.5509999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:10:52.980 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T09:10:52.980 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:10:53.327 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:53.328 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:10:53.345 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-13T09:10:53.112942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":336,"num_read":210966,"num_read_kb":690614,"num_write":25203,"num_write_kb":340526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21540,"ondisk_log_size":21540,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406292,"kb_used_data":20952,"kb_used_omap":30,"kb_used_meta":385185,"kb_avail":754568428,"statfs":{"total":773094113280,"available":772678070272,"internally_reserved":0,"allocated":21454848,"data_stored":10992532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31401,"internal_metadata":394429783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002368"},"pg_stats":[{"pgid":"3.1f","version":"190'569","reported_seq":5735,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001506+0000","last_change":"2024-09-13T09:10:27.358289+0000","last_active":"2024-09-13T09:10:46.001506+0000","last_peered":"2024-09-13T09:10:46.001506+0000","last_clean":"2024-09-13T09:10:46.001506+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:46.001506+0000","last_undegraded":"2024-09-13T09:10:46.001506+0000","last_fullsized":"2024-09-13T09:10:46.001506+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016218700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5971,"num_read_kb":20218,"num_write":677,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"189'1226","reported_seq":5143,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001090+0000","last_change":"2024-09-13T09:10:27.357623+0000","last_active":"2024-09-13T09:10:46.001090+0000","last_peered":"2024-09-13T09:10:46.001090+0000","last_clean":"2024-09-13T09:10:46.001090+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:10:46.001090+0000","last_undegraded":"2024-09-13T09:10:46.001090+0000","last_fullsized":"2024-09-13T09:10:46.001090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082279900000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9326,"num_read_kb":19231,"num_write":1571,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"188'1409","reported_seq":5692,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935670+0000","last_change":"2024-09-13T09:10:37.937627+0000","last_active":"2024-09-13T09:10:46.935670+0000","last_peered":"2024-09-13T09:10:46.935670+0000","last_clean":"2024-09-13T09:10:46.935670+0000","last_became_active":"2024-09-13T09:10:37.936189+0000","last_became_peered":"2024-09-13T09:10:37.936189+0000","last_unstale":"2024-09-13T09:10:46.935670+0000","last_undegraded":"2024-09-13T09:10:46.935670+0000","last_fullsized":"2024-09-13T09:10:46.935670+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:43.640432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013860960000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8733,"num_read_kb":34149,"num_write":2234,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"190'732","reported_seq":4385,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435424+0000","last_change":"2024-09-13T09:10:36.914556+0000","last_active":"2024-09-13T09:10:46.435424+0000","last_peered":"2024-09-13T09:10:46.435424+0000","last_clean":"2024-09-13T09:10:46.435424+0000","last_became_active":"2024-09-13T09:10:36.913240+0000","last_became_peered":"2024-09-13T09:10:36.913240+0000","last_unstale":"2024-09-13T09:10:46.435424+0000","last_undegraded":"2024-09-13T09:10:46.435424+0000","last_fullsized":"2024-09-13T09:10:46.435424+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:35:19.776639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015845119999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5745,"num_read_kb":35631,"num_write":972,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"188'730","reported_seq":22464,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936232+0000","last_change":"2024-09-13T09:10:36.919210+0000","last_active":"2024-09-13T09:10:46.936232+0000","last_peered":"2024-09-13T09:10:46.936232+0000","last_clean":"2024-09-13T09:10:46.936232+0000","last_became_active":"2024-09-13T09:10:36.914093+0000","last_became_peered":"2024-09-13T09:10:36.914093+0000","last_unstale":"2024-09-13T09:10:46.936232+0000","last_undegraded":"2024-09-13T09:10:46.936232+0000","last_fullsized":"2024-09-13T09:10:46.936232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:46.181581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023266060000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":23676,"num_read_kb":48180,"num_write":787,"num_write_kb":18573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"188'704","reported_seq":6536,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435442+0000","last_change":"2024-09-13T09:10:27.300037+0000","last_active":"2024-09-13T09:10:46.435442+0000","last_peered":"2024-09-13T09:10:46.435442+0000","last_clean":"2024-09-13T09:10:46.435442+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:10:46.435442+0000","last_undegraded":"2024-09-13T09:10:46.435442+0000","last_fullsized":"2024-09-13T09:10:46.435442+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:03:07.902590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093814799999999998,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":8455,"num_read_kb":25867,"num_write":848,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"188'387","reported_seq":2116,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935630+0000","last_change":"2024-09-13T09:10:27.328428+0000","last_active":"2024-09-13T09:10:46.935630+0000","last_peered":"2024-09-13T09:10:46.935630+0000","last_clean":"2024-09-13T09:10:46.935630+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:10:46.935630+0000","last_undegraded":"2024-09-13T09:10:46.935630+0000","last_fullsized":"2024-09-13T09:10:46.935630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340506,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1827,"num_read_kb":8140,"num_write":348,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"189'714","reported_seq":3577,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001432+0000","last_change":"2024-09-13T09:10:27.358518+0000","last_active":"2024-09-13T09:10:46.001432+0000","last_peered":"2024-09-13T09:10:46.001432+0000","last_clean":"2024-09-13T09:10:46.001432+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:10:46.001432+0000","last_undegraded":"2024-09-13T09:10:46.001432+0000","last_fullsized":"2024-09-13T09:10:46.001432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016018569999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4805,"num_read_kb":14792,"num_write":763,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"189'668","reported_seq":5966,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001614+0000","last_change":"2024-09-13T09:10:27.358098+0000","last_active":"2024-09-13T09:10:46.001614+0000","last_peered":"2024-09-13T09:10:46.001614+0000","last_clean":"2024-09-13T09:10:46.001614+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:10:46.001614+0000","last_undegraded":"2024-09-13T09:10:46.001614+0000","last_fullsized":"2024-09-13T09:10:46.001614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689800000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7350,"num_read_kb":31391,"num_write":638,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"189'718","reported_seq":2129,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424676+0000","last_change":"2024-09-13T09:10:27.295495+0000","last_active":"2024-09-13T09:10:46.424676+0000","last_peered":"2024-09-13T09:10:46.424676+0000","last_clean":"2024-09-13T09:10:46.424676+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:46.424676+0000","last_undegraded":"2024-09-13T09:10:46.424676+0000","last_fullsized":"2024-09-13T09:10:46.424676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:27.341075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020123239999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2936,"num_read_kb":14217,"num_write":967,"num_write_kb":9811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"189'734","reported_seq":4409,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002493+0000","last_change":"2024-09-13T09:10:36.923247+0000","last_active":"2024-09-13T09:10:46.002493+0000","last_peered":"2024-09-13T09:10:46.002493+0000","last_clean":"2024-09-13T09:10:46.002493+0000","last_became_active":"2024-09-13T09:10:36.918448+0000","last_became_peered":"2024-09-13T09:10:36.918448+0000","last_unstale":"2024-09-13T09:10:46.002493+0000","last_undegraded":"2024-09-13T09:10:46.002493+0000","last_fullsized":"2024-09-13T09:10:46.002493+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:10:36.185691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021014559999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5305,"num_read_kb":24795,"num_write":695,"num_write_kb":16167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"190'725","reported_seq":19435,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435181+0000","last_change":"2024-09-13T09:10:27.300467+0000","last_active":"2024-09-13T09:10:46.435181+0000","last_peered":"2024-09-13T09:10:46.435181+0000","last_clean":"2024-09-13T09:10:46.435181+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:10:46.435181+0000","last_undegraded":"2024-09-13T09:10:46.435181+0000","last_fullsized":"2024-09-13T09:10:46.435181+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:15:53.414704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012366219999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21128,"num_read_kb":34179,"num_write":635,"num_write_kb":10198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":221,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:10:33.349791+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:10:33.349249+0000","last_became_peered":"2024-09-13T09:10:33.349249+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:24.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"190'700","reported_seq":6003,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935995+0000","last_change":"2024-09-13T09:10:27.329077+0000","last_active":"2024-09-13T09:10:46.935995+0000","last_peered":"2024-09-13T09:10:46.935995+0000","last_clean":"2024-09-13T09:10:46.935995+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:10:46.935995+0000","last_undegraded":"2024-09-13T09:10:46.935995+0000","last_fullsized":"2024-09-13T09:10:46.935995+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:13.066567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028872860000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6169,"num_read_kb":21942,"num_write":789,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936143+0000","last_change":"2024-09-13T09:10:33.349800+0000","last_active":"2024-09-13T09:10:46.936143+0000","last_peered":"2024-09-13T09:10:46.936143+0000","last_clean":"2024-09-13T09:10:46.936143+0000","last_became_active":"2024-09-13T09:10:33.349192+0000","last_became_peered":"2024-09-13T09:10:33.349192+0000","last_unstale":"2024-09-13T09:10:46.936143+0000","last_undegraded":"2024-09-13T09:10:46.936143+0000","last_fullsized":"2024-09-13T09:10:46.936143+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:16.106956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"189'433","reported_seq":2543,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936105+0000","last_change":"2024-09-13T09:10:27.327294+0000","last_active":"2024-09-13T09:10:46.936105+0000","last_peered":"2024-09-13T09:10:46.936105+0000","last_clean":"2024-09-13T09:10:46.936105+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:10:46.936105+0000","last_undegraded":"2024-09-13T09:10:46.936105+0000","last_fullsized":"2024-09-13T09:10:46.936105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001256109,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2732,"num_read_kb":14647,"num_write":388,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"188'621","reported_seq":6246,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001146+0000","last_change":"2024-09-13T09:10:27.286574+0000","last_active":"2024-09-13T09:10:46.001146+0000","last_peered":"2024-09-13T09:10:46.001146+0000","last_clean":"2024-09-13T09:10:46.001146+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:10:46.001146+0000","last_undegraded":"2024-09-13T09:10:46.001146+0000","last_fullsized":"2024-09-13T09:10:46.001146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043649599999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6810,"num_read_kb":26019,"num_write":814,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001342+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:10:46.001342+0000","last_peered":"2024-09-13T09:10:46.001342+0000","last_clean":"2024-09-13T09:10:46.001342+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:10:46.001342+0000","last_undegraded":"2024-09-13T09:10:46.001342+0000","last_fullsized":"2024-09-13T09:10:46.001342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"189'717","reported_seq":3887,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424566+0000","last_change":"2024-09-13T09:10:27.294511+0000","last_active":"2024-09-13T09:10:46.424566+0000","last_peered":"2024-09-13T09:10:46.424566+0000","last_clean":"2024-09-13T09:10:46.424566+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:10:46.424566+0000","last_undegraded":"2024-09-13T09:10:46.424566+0000","last_fullsized":"2024-09-13T09:10:46.424566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:06:05.797639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014721929999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4939,"num_read_kb":19757,"num_write":904,"num_write_kb":10484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"190'667","reported_seq":3539,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002172+0000","last_change":"2024-09-13T09:10:36.919578+0000","last_active":"2024-09-13T09:10:46.002172+0000","last_peered":"2024-09-13T09:10:46.002172+0000","last_clean":"2024-09-13T09:10:46.002172+0000","last_became_active":"2024-09-13T09:10:36.917124+0000","last_became_peered":"2024-09-13T09:10:36.917124+0000","last_unstale":"2024-09-13T09:10:46.002172+0000","last_undegraded":"2024-09-13T09:10:46.002172+0000","last_fullsized":"2024-09-13T09:10:46.002172+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:11.427118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011757899999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3513,"num_read_kb":18416,"num_write":638,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":207,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935660+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:10:46.935660+0000","last_peered":"2024-09-13T09:10:46.935660+0000","last_clean":"2024-09-13T09:10:46.935660+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:10:46.935660+0000","last_undegraded":"2024-09-13T09:10:46.935660+0000","last_fullsized":"2024-09-13T09:10:46.935660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"189'504","reported_seq":3792,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001037+0000","last_change":"2024-09-13T09:10:27.357629+0000","last_active":"2024-09-13T09:10:46.001037+0000","last_peered":"2024-09-13T09:10:46.001037+0000","last_clean":"2024-09-13T09:10:46.001037+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:10:46.001037+0000","last_undegraded":"2024-09-13T09:10:46.001037+0000","last_fullsized":"2024-09-13T09:10:46.001037+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:27.395379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095688200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3585,"num_read_kb":19320,"num_write":505,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935744+0000","last_change":"2024-09-13T09:10:19.191965+0000","last_active":"2024-09-13T09:10:46.935744+0000","last_peered":"2024-09-13T09:10:46.935744+0000","last_clean":"2024-09-13T09:10:46.935744+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:46.935744+0000","last_undegraded":"2024-09-13T09:10:46.935744+0000","last_fullsized":"2024-09-13T09:10:46.935744+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:17.687266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"190'422","reported_seq":2354,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000648+0000","last_change":"2024-09-13T09:10:27.286775+0000","last_active":"2024-09-13T09:10:46.000648+0000","last_peered":"2024-09-13T09:10:46.000648+0000","last_clean":"2024-09-13T09:10:46.000648+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:10:46.000648+0000","last_undegraded":"2024-09-13T09:10:46.000648+0000","last_fullsized":"2024-09-13T09:10:46.000648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074152199999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2071,"num_read_kb":14898,"num_write":412,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935439+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:10:46.935439+0000","last_peered":"2024-09-13T09:10:46.935439+0000","last_clean":"2024-09-13T09:10:46.935439+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:10:46.935439+0000","last_undegraded":"2024-09-13T09:10:46.935439+0000","last_fullsized":"2024-09-13T09:10:46.935439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"189'405","reported_seq":2579,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936353+0000","last_change":"2024-09-13T09:10:27.326747+0000","last_active":"2024-09-13T09:10:46.936353+0000","last_peered":"2024-09-13T09:10:46.936353+0000","last_clean":"2024-09-13T09:10:46.936353+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:10:46.936353+0000","last_undegraded":"2024-09-13T09:10:46.936353+0000","last_fullsized":"2024-09-13T09:10:46.936353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037313,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2540,"num_read_kb":12233,"num_write":423,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935747+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:10:46.935747+0000","last_peered":"2024-09-13T09:10:46.935747+0000","last_clean":"2024-09-13T09:10:46.935747+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:10:46.935747+0000","last_undegraded":"2024-09-13T09:10:46.935747+0000","last_fullsized":"2024-09-13T09:10:46.935747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":335,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435002+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:10:46.435002+0000","last_peered":"2024-09-13T09:10:46.435002+0000","last_clean":"2024-09-13T09:10:46.435002+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:10:46.435002+0000","last_undegraded":"2024-09-13T09:10:46.435002+0000","last_fullsized":"2024-09-13T09:10:46.435002+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:40:36.564338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"190'881","reported_seq":6152,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935741+0000","last_change":"2024-09-13T09:10:27.327530+0000","last_active":"2024-09-13T09:10:46.935741+0000","last_peered":"2024-09-13T09:10:46.935741+0000","last_clean":"2024-09-13T09:10:46.935741+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:10:46.935741+0000","last_undegraded":"2024-09-13T09:10:46.935741+0000","last_fullsized":"2024-09-13T09:10:46.935741+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:33.786643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018329830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6854,"num_read_kb":23202,"num_write":1277,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"189'594","reported_seq":3645,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935677+0000","last_change":"2024-09-13T09:10:27.288793+0000","last_active":"2024-09-13T09:10:46.935677+0000","last_peered":"2024-09-13T09:10:46.935677+0000","last_clean":"2024-09-13T09:10:46.935677+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:46.935677+0000","last_undegraded":"2024-09-13T09:10:46.935677+0000","last_fullsized":"2024-09-13T09:10:46.935677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099698400000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4866,"num_read_kb":16848,"num_write":688,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"188'590","reported_seq":3110,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936333+0000","last_change":"2024-09-13T09:10:27.326271+0000","last_active":"2024-09-13T09:10:46.936333+0000","last_peered":"2024-09-13T09:10:46.936333+0000","last_clean":"2024-09-13T09:10:46.936333+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:46.936333+0000","last_undegraded":"2024-09-13T09:10:46.936333+0000","last_fullsized":"2024-09-13T09:10:46.936333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074287699999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4001,"num_read_kb":22630,"num_write":547,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"188'428","reported_seq":3878,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424141+0000","last_change":"2024-09-13T09:10:27.294587+0000","last_active":"2024-09-13T09:10:46.424141+0000","last_peered":"2024-09-13T09:10:46.424141+0000","last_clean":"2024-09-13T09:10:46.424141+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:10:46.424141+0000","last_undegraded":"2024-09-13T09:10:46.424141+0000","last_fullsized":"2024-09-13T09:10:46.424141+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:11.164172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013662279999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3594,"num_read_kb":33920,"num_write":421,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"187'360","reported_seq":17058,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002334+0000","last_change":"2024-09-13T09:10:35.921897+0000","last_active":"2024-09-13T09:10:46.002334+0000","last_peered":"2024-09-13T09:10:46.002334+0000","last_clean":"2024-09-13T09:10:46.002334+0000","last_became_active":"2024-09-13T09:10:35.919376+0000","last_became_peered":"2024-09-13T09:10:35.919376+0000","last_unstale":"2024-09-13T09:10:46.002334+0000","last_undegraded":"2024-09-13T09:10:46.002334+0000","last_fullsized":"2024-09-13T09:10:46.002334+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:08.251141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088103200000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":16596,"num_read_kb":27933,"num_write":361,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"190'410","reported_seq":3137,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935350+0000","last_change":"2024-09-13T09:10:27.289763+0000","last_active":"2024-09-13T09:10:46.935350+0000","last_peered":"2024-09-13T09:10:46.935350+0000","last_clean":"2024-09-13T09:10:46.935350+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:46.935350+0000","last_undegraded":"2024-09-13T09:10:46.935350+0000","last_fullsized":"2024-09-13T09:10:46.935350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001690069,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3095,"num_read_kb":9820,"num_write":360,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"190'884","reported_seq":7372,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.424756+0000","last_change":"2024-09-13T09:10:35.916276+0000","last_active":"2024-09-13T09:10:46.424756+0000","last_peered":"2024-09-13T09:10:46.424756+0000","last_clean":"2024-09-13T09:10:46.424756+0000","last_became_active":"2024-09-13T09:10:35.914456+0000","last_became_peered":"2024-09-13T09:10:35.914456+0000","last_unstale":"2024-09-13T09:10:46.424756+0000","last_undegraded":"2024-09-13T09:10:46.424756+0000","last_fullsized":"2024-09-13T09:10:46.424756+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:56:13.805979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001663692,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9292,"num_read_kb":23776,"num_write":1223,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"189'477","reported_seq":5178,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.002164+0000","last_change":"2024-09-13T09:10:35.918190+0000","last_active":"2024-09-13T09:10:46.002164+0000","last_peered":"2024-09-13T09:10:46.002164+0000","last_clean":"2024-09-13T09:10:46.002164+0000","last_became_active":"2024-09-13T09:10:35.915413+0000","last_became_peered":"2024-09-13T09:10:35.915413+0000","last_unstale":"2024-09-13T09:10:46.002164+0000","last_undegraded":"2024-09-13T09:10:46.002164+0000","last_fullsized":"2024-09-13T09:10:46.002164+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:23.432147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001550994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5016,"num_read_kb":17262,"num_write":425,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"188'518","reported_seq":2907,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.435248+0000","last_change":"2024-09-13T09:10:27.300360+0000","last_active":"2024-09-13T09:10:46.435248+0000","last_peered":"2024-09-13T09:10:46.435248+0000","last_clean":"2024-09-13T09:10:46.435248+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:10:46.435248+0000","last_undegraded":"2024-09-13T09:10:46.435248+0000","last_fullsized":"2024-09-13T09:10:46.435248+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:56.532022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011980339999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2903,"num_read_kb":17169,"num_write":491,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"190'1187","reported_seq":4411,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935485+0000","last_change":"2024-09-13T09:10:27.289049+0000","last_active":"2024-09-13T09:10:46.935485+0000","last_peered":"2024-09-13T09:10:46.935485+0000","last_clean":"2024-09-13T09:10:46.935485+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:46.935485+0000","last_undegraded":"2024-09-13T09:10:46.935485+0000","last_fullsized":"2024-09-13T09:10:46.935485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001456372,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7816,"num_read_kb":12367,"num_write":1664,"num_write_kb":5387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"189'718","reported_seq":4615,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000641+0000","last_change":"2024-09-13T09:10:27.287499+0000","last_active":"2024-09-13T09:10:46.000641+0000","last_peered":"2024-09-13T09:10:46.000641+0000","last_clean":"2024-09-13T09:10:46.000641+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:46.000641+0000","last_undegraded":"2024-09-13T09:10:46.000641+0000","last_fullsized":"2024-09-13T09:10:46.000641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012075230000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5826,"num_read_kb":14063,"num_write":971,"num_write_kb":11379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"187'588","reported_seq":4153,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935575+0000","last_change":"2024-09-13T09:10:31.329333+0000","last_active":"2024-09-13T09:10:46.935575+0000","last_peered":"2024-09-13T09:10:46.935575+0000","last_clean":"2024-09-13T09:10:46.935575+0000","last_became_active":"2024-09-13T09:10:31.325692+0000","last_became_peered":"2024-09-13T09:10:31.325692+0000","last_unstale":"2024-09-13T09:10:46.935575+0000","last_undegraded":"2024-09-13T09:10:46.935575+0000","last_fullsized":"2024-09-13T09:10:46.935575+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:57.571689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015796709999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3375,"num_read_kb":13502,"num_write":606,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":336,"num_read":210850,"num_read_kb":690514,"num_write":25042,"num_write_kb":338108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3692,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":21420,"ondisk_log_size":21420,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080585,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":2472,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65600,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306240,"statfs":{"total":96636764160,"available":96569589760,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64616904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.22,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.875,"5min":1.361,"15min":1.361},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63500000000000001,"15min":0.64700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.1459999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63600000000000001},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.88100000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.315}]},{"osd":2,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":1.444,"15min":1.444},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1850000000000001,"5min":1.766,"15min":1.766},"last":0.32900000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.745},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2170000000000001,"5min":1.343,"15min":1.343},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.919,"15min":1.919},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.90400000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4019999999999999,"5min":1.867,"15min":1.867},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.77000000000000002}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.56499999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69684,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302156,"statfs":{"total":96636764160,"available":96565407744,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.67300000000000004,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.226,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73099999999999998,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.024,"5min":1.353,"15min":1.353},"last":0.748}]},{"osd":1,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.2350000000000001,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.401,"5min":1.681,"15min":1.681},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.3,"5min":1.401,"15min":1.401},"last":0.92300000000000004}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":1.0249999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.288,"5min":1.998,"15min":1.998},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.391,"5min":1.651,"15min":1.651},"last":0.98999999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4379999999999999,"5min":1.506,"15min":1.506},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0009999999999999,"15min":1.0449999999999999},"min":{"1min":0.48899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.571,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.0660000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.071},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3500000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.089,"15min":1.125},"min":{"1min":0.621,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.417,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.94599999999999995}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.135},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":1.1060000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56316,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315524,"statfs":{"total":96636764160,"available":96579096576,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.30199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.86399999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.628}]},{"osd":2,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.624,"15min":0.64400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":1.054},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.90500000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.72399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0900000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3480000000000001,"5min":1.407,"15min":1.407},"last":0.95499999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3280000000000001,"5min":1.415,"15min":1.415},"last":1.4630000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.335,"5min":1.448,"15min":1.448},"last":1.145}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94399999999999995,"15min":0.97899999999999998},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":1.268},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1539999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.228}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93100000000000005,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4510000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.111},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.05},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.49,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.369},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.3029999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33588,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338252,"statfs":{"total":96636764160,"available":96602370048,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.59599999999999997},"min":{"1min":0.317,"5min":0.245,"15min":0.245},"max":{"1min":0.98099999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.625},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66000000000000003,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.79200000000000004,"15min":0.86699999999999999},"min":{"1min":0.38500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.149,"5min":3.855,"15min":3.855},"last":0.55400000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0309999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.496},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.82999999999999996,"15min":0.92600000000000005},"min":{"1min":0.40999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.341,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.58899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":0.93799999999999994},"min":{"1min":0.54800000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.218,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.753}]},{"osd":5,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.94099999999999995,"15min":0.97399999999999998},"min":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2949999999999999,"5min":2.532,"15min":2.532},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.997,"5min":1.0629999999999999,"15min":1.123},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.516,"5min":2.181,"15min":2.181},"last":0.97699999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.056,"15min":1.1240000000000001},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3700000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.018,"15min":1.073},"min":{"1min":0.63100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4910000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Sep 13 09:08:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.528,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0840000000000001,"15min":1.1220000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.74199999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542195,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37692,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334148,"statfs":{"total":96636764160,"available":96598167552,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.434,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1759999999999999,"5min":1.552,"15min":1.552},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.446,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.752}]},{"osd":1,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.79600000000000004},"min":{"1min":0.47799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2989999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":1.097},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.92000000000000004},"min":{"1min":0.53200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2669999999999999,"5min":2.706,"15min":2.706},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.93600000000000005,"15min":0.89700000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.649,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.84199999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.143},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.58899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1439999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.93400000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.022},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.92200000000000004,"15min":0.96399999999999997},"min":{"1min":0.498,"5min":0.377,"15min":0.377},"max":{"1min":1.5489999999999999,"5min":1.7,"15min":1.7},"last":1.202}]},{"osd":5,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.046,"15min":1.0800000000000001},"min":{"1min":0.5,"5min":0.497,"15min":0.497},"max":{"1min":1.359,"5min":1.776,"15min":1.776},"last":1.266},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.04,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.6220000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.323}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.4139999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.3520000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542194,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.121,"5min":2.649,"15min":2.649},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.202,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1819999999999999,"5min":1.3,"15min":1.3},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.88,"15min":0.83099999999999996},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.996},{"interface":"front","average":{"1min":1.042,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.03}]},{"osd":3,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.91600000000000004,"15min":0.86799999999999999},"min":{"1min":0.623,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.591,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.149},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.98999999999999999,"15min":0.98199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.90100000000000002,"15min":0.85799999999999998},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.008,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.95899999999999996,"15min":0.89000000000000001},"min":{"1min":0.54900000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.74,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.2450000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.98599999999999999,"15min":0.94699999999999995},"min":{"1min":0.44600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.042,"15min":1.0389999999999999},"min":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.21}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.284}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.032,"15min":1.1499999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1930000000000001,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":1.194}]},{"osd":1,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.83599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2490000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.335,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":1.0269999999999999,"15min":1.1319999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2809999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.1990000000000001},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3799999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":1.145},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.151,"5min":1.585,"15min":1.585},"last":1.034}]},{"osd":4,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.997,"15min":1.024},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4119999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.321},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.25,"15min":1.3799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.351,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.417}]},{"osd":6,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.97999999999999998,"15min":0.98399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.278,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.355,"5min":1.899,"15min":1.899},"last":1.518}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.5509999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:10:53.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T09:10:53.627 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:53.627 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":210,"fsid":"fe16bf6d-4166-4417-80fd-1123617a33d3","created":"2024-09-13T09:06:44.149006+0000","modified":"2024-09-13T09:10:47.004896+0000","last_up_change":"2024-09-13T09:10:18.167310+0000","last_in_change":"2024-09-13T09:10:26.268217+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T09:06:55.131330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T09:07:02.743102+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":21,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T09:07:12.517158+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":99,"snap_epoch":190,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33c6fc7d-9828-4500-b103-b96d1cd7f420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6818","nonce":3291756747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6822","nonce":3291756747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6830","nonce":3291756747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3291756747},{"type":"v1","addr":"172.21.15.31:6826","nonce":3291756747}]},"public_addr":"172.21.15.31:6818/3291756747","cluster_addr":"172.21.15.31:6822/3291756747","heartbeat_back_addr":"172.21.15.31:6830/3291756747","heartbeat_front_addr":"172.21.15.31:6826/3291756747","state":["exists","up"]},{"osd":1,"uuid":"869ea3c5-8e6e-4b49-b70e-083f242a7aae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6801","nonce":533520566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6803","nonce":533520566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6807","nonce":533520566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":533520566},{"type":"v1","addr":"172.21.15.31:6805","nonce":533520566}]},"public_addr":"172.21.15.31:6801/533520566","cluster_addr":"172.21.15.31:6803/533520566","heartbeat_back_addr":"172.21.15.31:6807/533520566","heartbeat_front_addr":"172.21.15.31:6805/533520566","state":["exists","up"]},{"osd":2,"uuid":"e144d496-76ee-46e0-bdc6-135c77128535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":198,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6817","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6819","nonce":2253604482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6821","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6823","nonce":2253604482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6829","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6831","nonce":2253604482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6825","nonce":2253604482},{"type":"v1","addr":"172.21.15.31:6827","nonce":2253604482}]},"public_addr":"172.21.15.31:6819/2253604482","cluster_addr":"172.21.15.31:6823/2253604482","heartbeat_back_addr":"172.21.15.31:6831/2253604482","heartbeat_front_addr":"172.21.15.31:6827/2253604482","state":["exists","up"]},{"osd":3,"uuid":"696b9414-9c19-4984-a8ff-14548e46b357","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6809","nonce":3367214693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6811","nonce":3367214693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6815","nonce":3367214693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3367214693},{"type":"v1","addr":"172.21.15.31:6813","nonce":3367214693}]},"public_addr":"172.21.15.31:6809/3367214693","cluster_addr":"172.21.15.31:6811/3367214693","heartbeat_back_addr":"172.21.15.31:6815/3367214693","heartbeat_front_addr":"172.21.15.31:6813/3367214693","state":["exists","up"]},{"osd":4,"uuid":"ac58040d-9b54-42ca-b143-1816abb54816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6801","nonce":3778074750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6803","nonce":3778074750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6807","nonce":3778074750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":3778074750},{"type":"v1","addr":"172.21.15.60:6805","nonce":3778074750}]},"public_addr":"172.21.15.60:6801/3778074750","cluster_addr":"172.21.15.60:6803/3778074750","heartbeat_back_addr":"172.21.15.60:6807/3778074750","heartbeat_front_addr":"172.21.15.60:6805/3778074750","state":["exists","up"]},{"osd":5,"uuid":"e56ca2b0-fe0e-4f16-88d8-cc476fd2d4e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6809","nonce":1926603087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6811","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6813","nonce":1926603087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6819","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6824","nonce":1926603087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6815","nonce":1926603087},{"type":"v1","addr":"172.21.15.60:6817","nonce":1926603087}]},"public_addr":"172.21.15.60:6809/1926603087","cluster_addr":"172.21.15.60:6813/1926603087","heartbeat_back_addr":"172.21.15.60:6824/1926603087","heartbeat_front_addr":"172.21.15.60:6817/1926603087","state":["exists","up"]},{"osd":6,"uuid":"d1c0a8d6-9346-4d44-9fa2-e15629b7ba4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6821","nonce":766305500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6823","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6825","nonce":766305500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6831","nonce":766305500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6827","nonce":766305500},{"type":"v1","addr":"172.21.15.60:6828","nonce":766305500}]},"public_addr":"172.21.15.60:6821/766305500","cluster_addr":"172.21.15.60:6825/766305500","heartbeat_back_addr":"172.21.15.60:6831/766305500","heartbeat_front_addr":"172.21.15.60:6828/766305500","state":["exists","up"]},{"osd":7,"uuid":"f36c332c-9497-47e8-b943-f6b6dcaebeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":157,"up_from":181,"up_thru":181,"down_at":158,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6812","nonce":3030128680}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6816","nonce":3030128680}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6829","nonce":3030128680}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3030128680},{"type":"v1","addr":"172.21.15.60:6822","nonce":3030128680}]},"public_addr":"172.21.15.60:6812/3030128680","cluster_addr":"172.21.15.60:6816/3030128680","heartbeat_back_addr":"172.21.15.60:6829/3030128680","heartbeat_front_addr":"172.21.15.60:6822/3030128680","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.183436+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.264080+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.253587+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.244429+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.369458+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.158089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.217295+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T09:06:52.332254+0000","dead_epoch":158}],"pg_upmap":[{"pgid":"3.a","osds":[4,2]},{"pgid":"3.b","osds":[2,1]}],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":2,"to":7}]},{"pgid":"3.14","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T09:10:54.645 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T09:10:54.645 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:10:54.809 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:54.809 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:10:54.809 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:54.826 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T09:10:55.113 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T09:10:55.132 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T09:10:55.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:10:55.279 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:55.280 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:10:55.280 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:55.295 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T09:10:55.584 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T09:10:55.604 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T09:10:55.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:10:55.758 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:55.758 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:10:55.758 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:55.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T09:10:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T09:10:56.084 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T09:10:56.085 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:10:56.236 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:56.237 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:10:56.237 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:56.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T09:10:56.547 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T09:10:56.567 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T09:10:56.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:10:56.715 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:56.716 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:10:56.716 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:56.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T09:10:57.022 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T09:10:57.041 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T09:10:57.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:10:57.192 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:57.192 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:10:57.192 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:57.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T09:10:57.509 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T09:10:57.528 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T09:10:57.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:10:57.670 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:57.670 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:10:57.671 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:57.685 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T09:10:57.961 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T09:10:57.978 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T09:10:57.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T09:10:58.126 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-13T09:10:58.127 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T09:10:58.127 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-13T09:10:58.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T09:10:58.435 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T09:10:58.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:10:58.761 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:10:58.763 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:10:58.779 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-13T09:10:57.114188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":336,"num_read":210966,"num_read_kb":690614,"num_write":25203,"num_write_kb":340526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21540,"ondisk_log_size":21540,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406292,"kb_used_data":20952,"kb_used_omap":30,"kb_used_meta":385185,"kb_avail":754568428,"statfs":{"total":773094113280,"available":772678070272,"internally_reserved":0,"allocated":21454848,"data_stored":10992532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31401,"internal_metadata":394429783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.098333"},"pg_stats":[{"pgid":"3.1f","version":"190'569","reported_seq":5735,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001506+0000","last_change":"2024-09-13T09:10:27.358289+0000","last_active":"2024-09-13T09:10:46.001506+0000","last_peered":"2024-09-13T09:10:46.001506+0000","last_clean":"2024-09-13T09:10:46.001506+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:46.001506+0000","last_undegraded":"2024-09-13T09:10:46.001506+0000","last_fullsized":"2024-09-13T09:10:46.001506+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:44:29.619786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016218700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5971,"num_read_kb":20218,"num_write":677,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"189'1226","reported_seq":5143,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001090+0000","last_change":"2024-09-13T09:10:27.357623+0000","last_active":"2024-09-13T09:10:46.001090+0000","last_peered":"2024-09-13T09:10:46.001090+0000","last_clean":"2024-09-13T09:10:46.001090+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:10:46.001090+0000","last_undegraded":"2024-09-13T09:10:46.001090+0000","last_fullsized":"2024-09-13T09:10:46.001090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:46.820464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082279900000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9326,"num_read_kb":19231,"num_write":1571,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"188'1409","reported_seq":5692,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935670+0000","last_change":"2024-09-13T09:10:37.937627+0000","last_active":"2024-09-13T09:10:46.935670+0000","last_peered":"2024-09-13T09:10:46.935670+0000","last_clean":"2024-09-13T09:10:46.935670+0000","last_became_active":"2024-09-13T09:10:37.936189+0000","last_became_peered":"2024-09-13T09:10:37.936189+0000","last_unstale":"2024-09-13T09:10:46.935670+0000","last_undegraded":"2024-09-13T09:10:46.935670+0000","last_fullsized":"2024-09-13T09:10:46.935670+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:46:43.640432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013860960000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8733,"num_read_kb":34149,"num_write":2234,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"190'732","reported_seq":4386,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.555273+0000","last_change":"2024-09-13T09:10:36.914556+0000","last_active":"2024-09-13T09:10:47.555273+0000","last_peered":"2024-09-13T09:10:47.555273+0000","last_clean":"2024-09-13T09:10:47.555273+0000","last_became_active":"2024-09-13T09:10:36.913240+0000","last_became_peered":"2024-09-13T09:10:36.913240+0000","last_unstale":"2024-09-13T09:10:47.555273+0000","last_undegraded":"2024-09-13T09:10:47.555273+0000","last_fullsized":"2024-09-13T09:10:47.555273+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":732,"log_dups_size":0,"ondisk_log_size":732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015845119999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5745,"num_read_kb":35631,"num_write":972,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"188'730","reported_seq":22464,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936232+0000","last_change":"2024-09-13T09:10:36.919210+0000","last_active":"2024-09-13T09:10:46.936232+0000","last_peered":"2024-09-13T09:10:46.936232+0000","last_clean":"2024-09-13T09:10:46.936232+0000","last_became_active":"2024-09-13T09:10:36.914093+0000","last_became_peered":"2024-09-13T09:10:36.914093+0000","last_unstale":"2024-09-13T09:10:46.936232+0000","last_undegraded":"2024-09-13T09:10:46.936232+0000","last_fullsized":"2024-09-13T09:10:46.936232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:04:46.181581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023266060000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":23676,"num_read_kb":48180,"num_write":787,"num_write_kb":18573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"188'704","reported_seq":6537,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.555153+0000","last_change":"2024-09-13T09:10:27.300037+0000","last_active":"2024-09-13T09:10:47.555153+0000","last_peered":"2024-09-13T09:10:47.555153+0000","last_clean":"2024-09-13T09:10:47.555153+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:10:47.555153+0000","last_undegraded":"2024-09-13T09:10:47.555153+0000","last_fullsized":"2024-09-13T09:10:47.555153+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":704,"log_dups_size":0,"ondisk_log_size":704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093814799999999998,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":8455,"num_read_kb":25867,"num_write":848,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"188'387","reported_seq":2116,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935630+0000","last_change":"2024-09-13T09:10:27.328428+0000","last_active":"2024-09-13T09:10:46.935630+0000","last_peered":"2024-09-13T09:10:46.935630+0000","last_clean":"2024-09-13T09:10:46.935630+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:10:46.935630+0000","last_undegraded":"2024-09-13T09:10:46.935630+0000","last_fullsized":"2024-09-13T09:10:46.935630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:19:11.693658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002340506,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1827,"num_read_kb":8140,"num_write":348,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"189'714","reported_seq":3577,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001432+0000","last_change":"2024-09-13T09:10:27.358518+0000","last_active":"2024-09-13T09:10:46.001432+0000","last_peered":"2024-09-13T09:10:46.001432+0000","last_clean":"2024-09-13T09:10:46.001432+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:10:46.001432+0000","last_undegraded":"2024-09-13T09:10:46.001432+0000","last_fullsized":"2024-09-13T09:10:46.001432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:12:46.497891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016018569999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4805,"num_read_kb":14792,"num_write":763,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"189'668","reported_seq":5966,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001614+0000","last_change":"2024-09-13T09:10:27.358098+0000","last_active":"2024-09-13T09:10:46.001614+0000","last_peered":"2024-09-13T09:10:46.001614+0000","last_clean":"2024-09-13T09:10:46.001614+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:10:46.001614+0000","last_undegraded":"2024-09-13T09:10:46.001614+0000","last_fullsized":"2024-09-13T09:10:46.001614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:22:59.715145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012689800000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7350,"num_read_kb":31391,"num_write":638,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"189'718","reported_seq":2130,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.554865+0000","last_change":"2024-09-13T09:10:27.295495+0000","last_active":"2024-09-13T09:10:47.554865+0000","last_peered":"2024-09-13T09:10:47.554865+0000","last_clean":"2024-09-13T09:10:47.554865+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:47.554865+0000","last_undegraded":"2024-09-13T09:10:47.554865+0000","last_fullsized":"2024-09-13T09:10:47.554865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020123239999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2936,"num_read_kb":14217,"num_write":967,"num_write_kb":9811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"189'734","reported_seq":4410,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.019638+0000","last_change":"2024-09-13T09:10:36.923247+0000","last_active":"2024-09-13T09:10:47.019638+0000","last_peered":"2024-09-13T09:10:47.019638+0000","last_clean":"2024-09-13T09:10:47.019638+0000","last_became_active":"2024-09-13T09:10:36.918448+0000","last_became_peered":"2024-09-13T09:10:36.918448+0000","last_unstale":"2024-09-13T09:10:47.019638+0000","last_undegraded":"2024-09-13T09:10:47.019638+0000","last_fullsized":"2024-09-13T09:10:47.019638+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":734,"log_dups_size":0,"ondisk_log_size":734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021014559999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5305,"num_read_kb":24795,"num_write":695,"num_write_kb":16167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"190'725","reported_seq":19436,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.555224+0000","last_change":"2024-09-13T09:10:27.300467+0000","last_active":"2024-09-13T09:10:47.555224+0000","last_peered":"2024-09-13T09:10:47.555224+0000","last_clean":"2024-09-13T09:10:47.555224+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:10:47.555224+0000","last_undegraded":"2024-09-13T09:10:47.555224+0000","last_fullsized":"2024-09-13T09:10:47.555224+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012366219999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21128,"num_read_kb":34179,"num_write":635,"num_write_kb":10198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":221,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:10:33.349791+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:10:33.349249+0000","last_became_peered":"2024-09-13T09:10:33.349249+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:38:24.714938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"190'700","reported_seq":6003,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935995+0000","last_change":"2024-09-13T09:10:27.329077+0000","last_active":"2024-09-13T09:10:46.935995+0000","last_peered":"2024-09-13T09:10:46.935995+0000","last_clean":"2024-09-13T09:10:46.935995+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:10:46.935995+0000","last_undegraded":"2024-09-13T09:10:46.935995+0000","last_fullsized":"2024-09-13T09:10:46.935995+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:08:13.066567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028872860000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6169,"num_read_kb":21942,"num_write":789,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":220,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936143+0000","last_change":"2024-09-13T09:10:33.349800+0000","last_active":"2024-09-13T09:10:46.936143+0000","last_peered":"2024-09-13T09:10:46.936143+0000","last_clean":"2024-09-13T09:10:46.936143+0000","last_became_active":"2024-09-13T09:10:33.349192+0000","last_became_peered":"2024-09-13T09:10:33.349192+0000","last_unstale":"2024-09-13T09:10:46.936143+0000","last_undegraded":"2024-09-13T09:10:46.936143+0000","last_fullsized":"2024-09-13T09:10:46.936143+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:21:16.106956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"189'433","reported_seq":2543,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936105+0000","last_change":"2024-09-13T09:10:27.327294+0000","last_active":"2024-09-13T09:10:46.936105+0000","last_peered":"2024-09-13T09:10:46.936105+0000","last_clean":"2024-09-13T09:10:46.936105+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:10:46.936105+0000","last_undegraded":"2024-09-13T09:10:46.936105+0000","last_fullsized":"2024-09-13T09:10:46.936105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:12.570199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001256109,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2732,"num_read_kb":14647,"num_write":388,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936023+0000","last_change":"2024-09-13T09:07:06.151545+0000","last_active":"2024-09-13T09:10:46.936023+0000","last_peered":"2024-09-13T09:10:46.936023+0000","last_clean":"2024-09-13T09:10:46.936023+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:10:46.936023+0000","last_undegraded":"2024-09-13T09:10:46.936023+0000","last_fullsized":"2024-09-13T09:10:46.936023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:57:54.916951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"188'621","reported_seq":6246,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001146+0000","last_change":"2024-09-13T09:10:27.286574+0000","last_active":"2024-09-13T09:10:46.001146+0000","last_peered":"2024-09-13T09:10:46.001146+0000","last_clean":"2024-09-13T09:10:46.001146+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:10:46.001146+0000","last_undegraded":"2024-09-13T09:10:46.001146+0000","last_fullsized":"2024-09-13T09:10:46.001146+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:47.613180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043649599999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6810,"num_read_kb":26019,"num_write":814,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001342+0000","last_change":"2024-09-13T09:07:06.239363+0000","last_active":"2024-09-13T09:10:46.001342+0000","last_peered":"2024-09-13T09:10:46.001342+0000","last_clean":"2024-09-13T09:10:46.001342+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:10:46.001342+0000","last_undegraded":"2024-09-13T09:10:46.001342+0000","last_fullsized":"2024-09-13T09:10:46.001342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:18:35.048642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"189'717","reported_seq":3888,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.554795+0000","last_change":"2024-09-13T09:10:27.294511+0000","last_active":"2024-09-13T09:10:47.554795+0000","last_peered":"2024-09-13T09:10:47.554795+0000","last_clean":"2024-09-13T09:10:47.554795+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:10:47.554795+0000","last_undegraded":"2024-09-13T09:10:47.554795+0000","last_fullsized":"2024-09-13T09:10:47.554795+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014721929999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4939,"num_read_kb":19757,"num_write":904,"num_write_kb":10484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"190'667","reported_seq":3540,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.019306+0000","last_change":"2024-09-13T09:10:36.919578+0000","last_active":"2024-09-13T09:10:47.019306+0000","last_peered":"2024-09-13T09:10:47.019306+0000","last_clean":"2024-09-13T09:10:47.019306+0000","last_became_active":"2024-09-13T09:10:36.917124+0000","last_became_peered":"2024-09-13T09:10:36.917124+0000","last_unstale":"2024-09-13T09:10:47.019306+0000","last_undegraded":"2024-09-13T09:10:47.019306+0000","last_fullsized":"2024-09-13T09:10:47.019306+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011757899999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3513,"num_read_kb":18416,"num_write":638,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"21'2","reported_seq":207,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935660+0000","last_change":"2024-09-13T09:07:06.150885+0000","last_active":"2024-09-13T09:10:46.935660+0000","last_peered":"2024-09-13T09:10:46.935660+0000","last_clean":"2024-09-13T09:10:46.935660+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:10:46.935660+0000","last_undegraded":"2024-09-13T09:10:46.935660+0000","last_fullsized":"2024-09-13T09:10:46.935660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:05.180495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"189'504","reported_seq":3792,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.001037+0000","last_change":"2024-09-13T09:10:27.357629+0000","last_active":"2024-09-13T09:10:46.001037+0000","last_peered":"2024-09-13T09:10:46.001037+0000","last_clean":"2024-09-13T09:10:46.001037+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:10:46.001037+0000","last_undegraded":"2024-09-13T09:10:46.001037+0000","last_fullsized":"2024-09-13T09:10:46.001037+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:27.395379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095688200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3585,"num_read_kb":19320,"num_write":505,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935744+0000","last_change":"2024-09-13T09:10:19.191965+0000","last_active":"2024-09-13T09:10:46.935744+0000","last_peered":"2024-09-13T09:10:46.935744+0000","last_clean":"2024-09-13T09:10:46.935744+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:46.935744+0000","last_undegraded":"2024-09-13T09:10:46.935744+0000","last_fullsized":"2024-09-13T09:10:46.935744+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:34:17.687266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"190'422","reported_seq":2354,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000648+0000","last_change":"2024-09-13T09:10:27.286775+0000","last_active":"2024-09-13T09:10:46.000648+0000","last_peered":"2024-09-13T09:10:46.000648+0000","last_clean":"2024-09-13T09:10:46.000648+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:10:46.000648+0000","last_undegraded":"2024-09-13T09:10:46.000648+0000","last_fullsized":"2024-09-13T09:10:46.000648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:21:43.495493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074152199999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2071,"num_read_kb":14898,"num_write":412,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935439+0000","last_change":"2024-09-13T09:07:06.151050+0000","last_active":"2024-09-13T09:10:46.935439+0000","last_peered":"2024-09-13T09:10:46.935439+0000","last_clean":"2024-09-13T09:10:46.935439+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:10:46.935439+0000","last_undegraded":"2024-09-13T09:10:46.935439+0000","last_fullsized":"2024-09-13T09:10:46.935439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:50:29.142347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"189'405","reported_seq":2579,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936353+0000","last_change":"2024-09-13T09:10:27.326747+0000","last_active":"2024-09-13T09:10:46.936353+0000","last_peered":"2024-09-13T09:10:46.936353+0000","last_clean":"2024-09-13T09:10:46.936353+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:10:46.936353+0000","last_undegraded":"2024-09-13T09:10:46.936353+0000","last_fullsized":"2024-09-13T09:10:46.936353+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:25.939624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001037313,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2540,"num_read_kb":12233,"num_write":423,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"19'1","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935747+0000","last_change":"2024-09-13T09:07:06.149435+0000","last_active":"2024-09-13T09:10:46.935747+0000","last_peered":"2024-09-13T09:10:46.935747+0000","last_clean":"2024-09-13T09:10:46.935747+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:10:46.935747+0000","last_undegraded":"2024-09-13T09:10:46.935747+0000","last_fullsized":"2024-09-13T09:10:46.935747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:03.102627+0000","last_clean_scrub_stamp":"2024-09-13T09:07:03.102627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:15:04.078010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":336,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.555299+0000","last_change":"2024-09-13T09:07:53.862191+0000","last_active":"2024-09-13T09:10:47.555299+0000","last_peered":"2024-09-13T09:10:47.555299+0000","last_clean":"2024-09-13T09:10:47.555299+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:10:47.555299+0000","last_undegraded":"2024-09-13T09:10:47.555299+0000","last_fullsized":"2024-09-13T09:10:47.555299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:06:55.611487+0000","last_clean_scrub_stamp":"2024-09-13T09:06:55.611487+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"190'881","reported_seq":6152,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935741+0000","last_change":"2024-09-13T09:10:27.327530+0000","last_active":"2024-09-13T09:10:46.935741+0000","last_peered":"2024-09-13T09:10:46.935741+0000","last_clean":"2024-09-13T09:10:46.935741+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:10:46.935741+0000","last_undegraded":"2024-09-13T09:10:46.935741+0000","last_fullsized":"2024-09-13T09:10:46.935741+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:30:33.786643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018329830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6854,"num_read_kb":23202,"num_write":1277,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"189'594","reported_seq":3645,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935677+0000","last_change":"2024-09-13T09:10:27.288793+0000","last_active":"2024-09-13T09:10:46.935677+0000","last_peered":"2024-09-13T09:10:46.935677+0000","last_clean":"2024-09-13T09:10:46.935677+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:46.935677+0000","last_undegraded":"2024-09-13T09:10:46.935677+0000","last_fullsized":"2024-09-13T09:10:46.935677+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:39.966119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099698400000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4866,"num_read_kb":16848,"num_write":688,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"188'590","reported_seq":3110,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.936333+0000","last_change":"2024-09-13T09:10:27.326271+0000","last_active":"2024-09-13T09:10:46.936333+0000","last_peered":"2024-09-13T09:10:46.936333+0000","last_clean":"2024-09-13T09:10:46.936333+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:46.936333+0000","last_undegraded":"2024-09-13T09:10:46.936333+0000","last_fullsized":"2024-09-13T09:10:46.936333+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:10:28.791869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074287699999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4001,"num_read_kb":22630,"num_write":547,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"188'428","reported_seq":3879,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.554892+0000","last_change":"2024-09-13T09:10:27.294587+0000","last_active":"2024-09-13T09:10:47.554892+0000","last_peered":"2024-09-13T09:10:47.554892+0000","last_clean":"2024-09-13T09:10:47.554892+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:10:47.554892+0000","last_undegraded":"2024-09-13T09:10:47.554892+0000","last_fullsized":"2024-09-13T09:10:47.554892+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013662279999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3594,"num_read_kb":33920,"num_write":421,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"187'360","reported_seq":17059,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.019535+0000","last_change":"2024-09-13T09:10:35.921897+0000","last_active":"2024-09-13T09:10:47.019535+0000","last_peered":"2024-09-13T09:10:47.019535+0000","last_clean":"2024-09-13T09:10:47.019535+0000","last_became_active":"2024-09-13T09:10:35.919376+0000","last_became_peered":"2024-09-13T09:10:35.919376+0000","last_unstale":"2024-09-13T09:10:47.019535+0000","last_undegraded":"2024-09-13T09:10:47.019535+0000","last_fullsized":"2024-09-13T09:10:47.019535+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088103200000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":16596,"num_read_kb":27933,"num_write":361,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"190'410","reported_seq":3137,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935350+0000","last_change":"2024-09-13T09:10:27.289763+0000","last_active":"2024-09-13T09:10:46.935350+0000","last_peered":"2024-09-13T09:10:46.935350+0000","last_clean":"2024-09-13T09:10:46.935350+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:46.935350+0000","last_undegraded":"2024-09-13T09:10:46.935350+0000","last_fullsized":"2024-09-13T09:10:46.935350+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:50:09.752016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001690069,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3095,"num_read_kb":9820,"num_write":360,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"190'884","reported_seq":7373,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.554949+0000","last_change":"2024-09-13T09:10:35.916276+0000","last_active":"2024-09-13T09:10:47.554949+0000","last_peered":"2024-09-13T09:10:47.554949+0000","last_clean":"2024-09-13T09:10:47.554949+0000","last_became_active":"2024-09-13T09:10:35.914456+0000","last_became_peered":"2024-09-13T09:10:35.914456+0000","last_unstale":"2024-09-13T09:10:47.554949+0000","last_undegraded":"2024-09-13T09:10:47.554949+0000","last_fullsized":"2024-09-13T09:10:47.554949+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001663692,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9292,"num_read_kb":23776,"num_write":1223,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"189'477","reported_seq":5179,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.019637+0000","last_change":"2024-09-13T09:10:35.918190+0000","last_active":"2024-09-13T09:10:47.019637+0000","last_peered":"2024-09-13T09:10:47.019637+0000","last_clean":"2024-09-13T09:10:47.019637+0000","last_became_active":"2024-09-13T09:10:35.915413+0000","last_became_peered":"2024-09-13T09:10:35.915413+0000","last_unstale":"2024-09-13T09:10:47.019637+0000","last_undegraded":"2024-09-13T09:10:47.019637+0000","last_fullsized":"2024-09-13T09:10:47.019637+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001550994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5016,"num_read_kb":17262,"num_write":425,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"188'518","reported_seq":2908,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.555130+0000","last_change":"2024-09-13T09:10:27.300360+0000","last_active":"2024-09-13T09:10:47.555130+0000","last_peered":"2024-09-13T09:10:47.555130+0000","last_clean":"2024-09-13T09:10:47.555130+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:10:47.555130+0000","last_undegraded":"2024-09-13T09:10:47.555130+0000","last_fullsized":"2024-09-13T09:10:47.555130+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011980339999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2903,"num_read_kb":17169,"num_write":491,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"190'1187","reported_seq":4411,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935485+0000","last_change":"2024-09-13T09:10:27.289049+0000","last_active":"2024-09-13T09:10:46.935485+0000","last_peered":"2024-09-13T09:10:46.935485+0000","last_clean":"2024-09-13T09:10:46.935485+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:46.935485+0000","last_undegraded":"2024-09-13T09:10:46.935485+0000","last_fullsized":"2024-09-13T09:10:46.935485+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:01:13.830257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001456372,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7816,"num_read_kb":12367,"num_write":1664,"num_write_kb":5387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"189'718","reported_seq":4615,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.000641+0000","last_change":"2024-09-13T09:10:27.287499+0000","last_active":"2024-09-13T09:10:46.000641+0000","last_peered":"2024-09-13T09:10:46.000641+0000","last_clean":"2024-09-13T09:10:46.000641+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:46.000641+0000","last_undegraded":"2024-09-13T09:10:46.000641+0000","last_fullsized":"2024-09-13T09:10:46.000641+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:50:45.101982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012075230000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5826,"num_read_kb":14063,"num_write":971,"num_write_kb":11379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"187'588","reported_seq":4153,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-13T09:10:46.935575+0000","last_change":"2024-09-13T09:10:31.329333+0000","last_active":"2024-09-13T09:10:46.935575+0000","last_peered":"2024-09-13T09:10:46.935575+0000","last_clean":"2024-09-13T09:10:46.935575+0000","last_became_active":"2024-09-13T09:10:31.325692+0000","last_became_peered":"2024-09-13T09:10:31.325692+0000","last_unstale":"2024-09-13T09:10:46.935575+0000","last_undegraded":"2024-09-13T09:10:46.935575+0000","last_fullsized":"2024-09-13T09:10:46.935575+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:07:57.571689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015796709999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3375,"num_read_kb":13502,"num_write":606,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":336,"num_read":210850,"num_read_kb":690514,"num_write":25042,"num_write_kb":338108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3692,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":21420,"ondisk_log_size":21420,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080585,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":2472,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509492,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65600,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306240,"statfs":{"total":96636764160,"available":96569589760,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":64616904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.22,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.875,"5min":1.361,"15min":1.361},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.63500000000000001,"15min":0.64700000000000002},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.1459999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.86099999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61099999999999999,"15min":0.63600000000000001},"min":{"1min":0.27100000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.88100000000000001,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92600000000000005,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.627,"15min":0.629},"min":{"1min":0.30599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":1.0469999999999999,"5min":1.444,"15min":1.444},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.57699999999999996},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.1850000000000001,"5min":1.766,"15min":1.766},"last":0.89200000000000002}]},{"osd":4,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.42599999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.2170000000000001,"5min":1.343,"15min":1.343},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.86199999999999999,"15min":0.86099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3180000000000001,"5min":1.919,"15min":1.919},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82499999999999996},"min":{"1min":0.51400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.95999999999999996}]},{"osd":6,"last update":"Fri Sep 13 09:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.91500000000000004,"15min":0.90400000000000003},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.4019999999999999,"5min":1.867,"15min":1.867},"last":1.349},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.91100000000000003,"15min":0.91600000000000004},"min":{"1min":0.56599999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":1.05}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":1.135},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.393}]}]},{"osd":2,"up_from":15,"seq":64424509491,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69684,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302156,"statfs":{"total":96636764160,"available":96565407744,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.67300000000000004,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.226,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73099999999999998,"15min":0.748},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.024,"5min":1.353,"15min":1.353},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.501},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.78000000000000003,"15min":0.79000000000000004},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.2350000000000001,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.401,"5min":1.681,"15min":1.681},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.84699999999999998,"15min":0.90300000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":1.3,"5min":1.401,"15min":1.401},"last":0.754}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.96899999999999997,"15min":1.0249999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.288,"5min":1.998,"15min":1.998},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.91600000000000004,"15min":0.94799999999999995},"min":{"1min":0.52200000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.391,"5min":1.651,"15min":1.651},"last":0.94099999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.96199999999999997,"15min":0.98199999999999998},"min":{"1min":0.53900000000000003,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4379999999999999,"5min":1.506,"15min":1.506},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0009999999999999,"15min":1.0449999999999999},"min":{"1min":0.48899999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.571,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.98199999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.0249999999999999,"15min":1.071},"min":{"1min":0.5,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3500000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.089,"15min":1.125},"min":{"1min":0.621,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.417,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.80700000000000005}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.069},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":1.022}]}]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56316,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315524,"statfs":{"total":96636764160,"available":96579096576,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.58899999999999997,"15min":0.59699999999999998},"min":{"1min":0.30199999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.86399999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.61499999999999999,"15min":0.64100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.41999999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.624,"15min":0.64400000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.95999999999999996,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.33600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.90800000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69199999999999995,"15min":0.72399999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0900000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.76000000000000001},"min":{"1min":0.35699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3480000000000001,"5min":1.407,"15min":1.407},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.82999999999999996,"15min":0.84299999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.3280000000000001,"5min":1.415,"15min":1.415},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.90100000000000002,"15min":0.93500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.335,"5min":1.448,"15min":1.448},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.94399999999999995,"15min":0.97899999999999998},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":1.3879999999999999,"5min":2.0299999999999998,"15min":2.0299999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92900000000000005,"15min":0.95699999999999996},"min":{"1min":0.54500000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1539999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:00 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.93100000000000005,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.4510000000000001,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.97499999999999998,"5min":1.012,"15min":1.05},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.49,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.141}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":0.88500000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33588,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338252,"statfs":{"total":96636764160,"available":96602370048,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56999999999999995,"15min":0.59599999999999997},"min":{"1min":0.317,"5min":0.245,"15min":0.245},"max":{"1min":0.98099999999999998,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.625},"min":{"1min":0.26400000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.66000000000000003,"15min":0.69999999999999996},"min":{"1min":0.32700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.79200000000000004,"15min":0.86699999999999999},"min":{"1min":0.38500000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.149,"5min":3.855,"15min":3.855},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.75,"15min":0.81799999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0309999999999999,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.82999999999999996,"15min":0.92600000000000005},"min":{"1min":0.40999999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.341,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.877,"15min":0.88700000000000001},"min":{"1min":0.58899999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2629999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.89300000000000002,"15min":0.93799999999999994},"min":{"1min":0.54800000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.218,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.94099999999999995,"15min":0.97399999999999998},"min":{"1min":0.55900000000000005,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.2949999999999999,"5min":2.532,"15min":2.532},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.997,"5min":1.0629999999999999,"15min":1.123},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.516,"5min":2.181,"15min":2.181},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":1.056,"15min":1.1240000000000001},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3700000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.018,"15min":1.073},"min":{"1min":0.63100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.4910000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:08:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.528,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0840000000000001,"15min":1.1220000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.625}]}]},{"osd":6,"up_from":14,"seq":60129542196,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37692,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334148,"statfs":{"total":96636764160,"available":96598167552,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79500000000000004,"15min":0.82099999999999995},"min":{"1min":0.434,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.1759999999999999,"5min":1.552,"15min":1.552},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.76800000000000002},"min":{"1min":0.50600000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.446,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.80100000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81899999999999995,"15min":0.79600000000000004},"min":{"1min":0.47799999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2989999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.90400000000000003,"15min":0.92000000000000004},"min":{"1min":0.53200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.2669999999999999,"5min":2.706,"15min":2.706},"last":0.92500000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.93600000000000005,"15min":0.89700000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.649,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.96699999999999997,"15min":0.93899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.274,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:10:05 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.93500000000000005,"15min":0.94799999999999995},"min":{"1min":0.55800000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.91900000000000004,"15min":0.92100000000000004},"min":{"1min":0.58899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1439999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.754}]},{"osd":4,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.97299999999999998,"15min":0.99199999999999999},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.92200000000000004,"15min":0.96399999999999997},"min":{"1min":0.498,"5min":0.377,"15min":0.377},"max":{"1min":1.5489999999999999,"5min":1.7,"15min":1.7},"last":0.88}]},{"osd":5,"last update":"Fri Sep 13 09:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.046,"15min":1.0800000000000001},"min":{"1min":0.5,"5min":0.497,"15min":0.497},"max":{"1min":1.359,"5min":1.776,"15min":1.776},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":1.04,"15min":1.0620000000000001},"min":{"1min":0.47299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.6220000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.131}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.089},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.008}]}]},{"osd":4,"up_from":14,"seq":60129542195,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63928,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94307912,"statfs":{"total":96636764160,"available":96571301888,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.84099999999999997},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.121,"5min":2.649,"15min":2.649},"last":0.71899999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.73899999999999999,"15min":0.73499999999999999},"min":{"1min":0.505,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.202,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1819999999999999,"5min":1.3,"15min":1.3},"last":0.89900000000000002}]},{"osd":2,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.88,"15min":0.83099999999999996},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.488,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.042,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.0960000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:09 2024","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.91600000000000004,"15min":0.86799999999999999},"min":{"1min":0.623,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.591,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":0.98999999999999999,"15min":0.98199999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.006}]},{"osd":5,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.90100000000000002,"15min":0.85799999999999998},"min":{"1min":0.42999999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":2.008,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.21,"5min":0.95899999999999996,"15min":0.89000000000000001},"min":{"1min":0.54900000000000004,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.74,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":1.4219999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":0.98599999999999999,"15min":0.94699999999999995},"min":{"1min":0.44600000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.161},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.042,"15min":1.0389999999999999},"min":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.476},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.2689999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79000000000000004,"15min":0.79500000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":1.032,"15min":1.1499999999999999},"min":{"1min":0.502,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1930000000000001,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":1.1850000000000001}]},{"osd":1,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.83599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2490000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51800000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.335,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":1.0269999999999999,"15min":1.1319999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2809999999999999,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.86699999999999999,"15min":0.88700000000000001},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.2,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.0940000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.0880000000000001,"15min":1.1990000000000001},"min":{"1min":0.495,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3799999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":1.008},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.94099999999999995,"15min":0.96899999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.151,"5min":1.585,"15min":1.585},"last":1.2609999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.997,"15min":1.024},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.4119999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":1.25,"15min":1.3799999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.351,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.1419999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:10:04 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.97999999999999998,"15min":0.98399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.278,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.391},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.048,"15min":1.0800000000000001},"min":{"1min":0.371,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.355,"5min":1.899,"15min":1.899},"last":1.359}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.3120000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:10:58.785 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.785 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.785 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.785 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.785 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-13T09:07:03.102627+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.786 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-13T09:07:03.102627+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-13T09:07:03.102627+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-13T09:07:03.102627+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-13T09:07:03.102627+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.787 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-13T09:07:03.102627+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T09:07:03.102627+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-13T09:07:03.102627+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=3, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T09:06:55.611487+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=6, tm_sec=55, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.788 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.789 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.789 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.789 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.789 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.789 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.789 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.789 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.789 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.790 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:10:58.790 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T09:11:18.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:11:19.111 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:11:19.112 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:11:19.129 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-13T09:11:17.121132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":336,"num_read":210966,"num_read_kb":690614,"num_write":25203,"num_write_kb":340526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21809,"ondisk_log_size":21809,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":398040,"kb_used_data":20952,"kb_used_omap":30,"kb_used_meta":376993,"kb_avail":754576680,"statfs":{"total":773094113280,"available":772686520320,"internally_reserved":0,"allocated":21454848,"data_stored":10992532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31401,"internal_metadata":386041175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003955"},"pg_stats":[{"pgid":"3.1f","version":"190'569","reported_seq":5736,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.019408+0000","last_change":"2024-09-13T09:10:27.358289+0000","last_active":"2024-09-13T09:10:47.019408+0000","last_peered":"2024-09-13T09:10:47.019408+0000","last_clean":"2024-09-13T09:10:47.019408+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:10:47.019408+0000","last_undegraded":"2024-09-13T09:10:47.019408+0000","last_fullsized":"2024-09-13T09:10:47.019408+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016218700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5971,"num_read_kb":20218,"num_write":677,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"210'1237","reported_seq":5167,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:10.834228+0000","last_change":"2024-09-13T09:11:10.834228+0000","last_active":"2024-09-13T09:11:10.834228+0000","last_peered":"2024-09-13T09:11:10.834228+0000","last_clean":"2024-09-13T09:11:10.834228+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:11:10.834228+0000","last_undegraded":"2024-09-13T09:11:10.834228+0000","last_fullsized":"2024-09-13T09:11:10.834228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'1237","last_scrub_stamp":"2024-09-13T09:11:10.833981+0000","last_deep_scrub":"210'1237","last_deep_scrub_stamp":"2024-09-13T09:11:10.833981+0000","last_clean_scrub_stamp":"2024-09-13T09:11:10.833981+0000","objects_scrubbed":11,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:27:13.914436+0000","scrub_duration":0.021379769,"objects_trimmed":0,"snaptrim_duration":0.00082279900000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9326,"num_read_kb":19231,"num_write":1571,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"188'1409","reported_seq":5693,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.468327+0000","last_change":"2024-09-13T09:10:37.937627+0000","last_active":"2024-09-13T09:10:47.468327+0000","last_peered":"2024-09-13T09:10:47.468327+0000","last_clean":"2024-09-13T09:10:47.468327+0000","last_became_active":"2024-09-13T09:10:37.936189+0000","last_became_peered":"2024-09-13T09:10:37.936189+0000","last_unstale":"2024-09-13T09:10:47.468327+0000","last_undegraded":"2024-09-13T09:10:47.468327+0000","last_fullsized":"2024-09-13T09:10:47.468327+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013860960000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8733,"num_read_kb":34149,"num_write":2234,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"210'742","reported_seq":4407,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:00.907957+0000","last_change":"2024-09-13T09:11:00.908182+0000","last_active":"2024-09-13T09:11:00.907957+0000","last_peered":"2024-09-13T09:11:00.907957+0000","last_clean":"2024-09-13T09:11:00.907957+0000","last_became_active":"2024-09-13T09:10:36.913240+0000","last_became_peered":"2024-09-13T09:10:36.913240+0000","last_unstale":"2024-09-13T09:11:00.907957+0000","last_undegraded":"2024-09-13T09:11:00.907957+0000","last_fullsized":"2024-09-13T09:11:00.907957+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'742","last_scrub_stamp":"2024-09-13T09:11:00.907901+0000","last_deep_scrub":"210'742","last_deep_scrub_stamp":"2024-09-13T09:11:00.907901+0000","last_clean_scrub_stamp":"2024-09-13T09:11:00.907901+0000","objects_scrubbed":10,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:41.042092+0000","scrub_duration":0.022088087999999999,"objects_trimmed":0,"snaptrim_duration":0.0015845119999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5745,"num_read_kb":35631,"num_write":972,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"210'738","reported_seq":22484,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:03.159232+0000","last_change":"2024-09-13T09:11:03.159453+0000","last_active":"2024-09-13T09:11:03.159232+0000","last_peered":"2024-09-13T09:11:03.159232+0000","last_clean":"2024-09-13T09:11:03.159232+0000","last_became_active":"2024-09-13T09:10:36.914093+0000","last_became_peered":"2024-09-13T09:10:36.914093+0000","last_unstale":"2024-09-13T09:11:03.159232+0000","last_undegraded":"2024-09-13T09:11:03.159232+0000","last_fullsized":"2024-09-13T09:11:03.159232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'738","last_scrub_stamp":"2024-09-13T09:11:03.159180+0000","last_deep_scrub":"210'738","last_deep_scrub_stamp":"2024-09-13T09:11:03.159180+0000","last_clean_scrub_stamp":"2024-09-13T09:11:03.159180+0000","objects_scrubbed":8,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:37.381128+0000","scrub_duration":0.015880679000000002,"objects_trimmed":0,"snaptrim_duration":0.0023266060000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":23676,"num_read_kb":48180,"num_write":787,"num_write_kb":18573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"210'718","reported_seq":6562,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:01.943284+0000","last_change":"2024-09-13T09:11:01.943676+0000","last_active":"2024-09-13T09:11:01.943284+0000","last_peered":"2024-09-13T09:11:01.943284+0000","last_clean":"2024-09-13T09:11:01.943284+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:11:01.943284+0000","last_undegraded":"2024-09-13T09:11:01.943284+0000","last_fullsized":"2024-09-13T09:11:01.943284+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'718","last_scrub_stamp":"2024-09-13T09:11:01.943177+0000","last_deep_scrub":"210'718","last_deep_scrub_stamp":"2024-09-13T09:11:01.943177+0000","last_clean_scrub_stamp":"2024-09-13T09:11:01.943177+0000","objects_scrubbed":14,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:22.134328+0000","scrub_duration":0.026873905999999999,"objects_trimmed":0,"snaptrim_duration":0.00093814799999999998,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":8455,"num_read_kb":25867,"num_write":848,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"210'401","reported_seq":2142,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:10.034913+0000","last_change":"2024-09-13T09:11:10.035158+0000","last_active":"2024-09-13T09:11:10.034913+0000","last_peered":"2024-09-13T09:11:10.034913+0000","last_clean":"2024-09-13T09:11:10.034913+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:11:10.034913+0000","last_undegraded":"2024-09-13T09:11:10.034913+0000","last_fullsized":"2024-09-13T09:11:10.034913+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'401","last_scrub_stamp":"2024-09-13T09:11:10.034858+0000","last_deep_scrub":"210'401","last_deep_scrub_stamp":"2024-09-13T09:11:10.034858+0000","last_clean_scrub_stamp":"2024-09-13T09:11:10.034858+0000","objects_scrubbed":14,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.863823+0000","scrub_duration":0.021874984,"objects_trimmed":0,"snaptrim_duration":0.002340506,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1827,"num_read_kb":8140,"num_write":348,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"210'725","reported_seq":3600,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:06.902907+0000","last_change":"2024-09-13T09:11:06.903138+0000","last_active":"2024-09-13T09:11:06.902907+0000","last_peered":"2024-09-13T09:11:06.902907+0000","last_clean":"2024-09-13T09:11:06.902907+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:11:06.902907+0000","last_undegraded":"2024-09-13T09:11:06.902907+0000","last_fullsized":"2024-09-13T09:11:06.902907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'725","last_scrub_stamp":"2024-09-13T09:11:06.902854+0000","last_deep_scrub":"210'725","last_deep_scrub_stamp":"2024-09-13T09:11:06.902854+0000","last_clean_scrub_stamp":"2024-09-13T09:11:06.902854+0000","objects_scrubbed":11,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:02:48.704124+0000","scrub_duration":0.021786298999999999,"objects_trimmed":0,"snaptrim_duration":0.0016018569999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4805,"num_read_kb":14792,"num_write":763,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"210'674","reported_seq":5984,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:07.858713+0000","last_change":"2024-09-13T09:11:07.859015+0000","last_active":"2024-09-13T09:11:07.858713+0000","last_peered":"2024-09-13T09:11:07.858713+0000","last_clean":"2024-09-13T09:11:07.858713+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:11:07.858713+0000","last_undegraded":"2024-09-13T09:11:07.858713+0000","last_fullsized":"2024-09-13T09:11:07.858713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'674","last_scrub_stamp":"2024-09-13T09:11:07.858660+0000","last_deep_scrub":"210'674","last_deep_scrub_stamp":"2024-09-13T09:11:07.858660+0000","last_clean_scrub_stamp":"2024-09-13T09:11:07.858660+0000","objects_scrubbed":6,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:50.834925+0000","scrub_duration":0.015685364,"objects_trimmed":0,"snaptrim_duration":0.0012689800000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7350,"num_read_kb":31391,"num_write":638,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"210'726","reported_seq":2149,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:58.509695+0000","last_change":"2024-09-13T09:10:58.509825+0000","last_active":"2024-09-13T09:10:58.509695+0000","last_peered":"2024-09-13T09:10:58.509695+0000","last_clean":"2024-09-13T09:10:58.509695+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:58.509695+0000","last_undegraded":"2024-09-13T09:10:58.509695+0000","last_fullsized":"2024-09-13T09:10:58.509695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'726","last_scrub_stamp":"2024-09-13T09:10:58.509653+0000","last_deep_scrub":"210'726","last_deep_scrub_stamp":"2024-09-13T09:10:58.509653+0000","last_clean_scrub_stamp":"2024-09-13T09:10:58.509653+0000","objects_scrubbed":8,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:02.988273+0000","scrub_duration":0.0086180700000000002,"objects_trimmed":0,"snaptrim_duration":0.0020123239999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2936,"num_read_kb":14217,"num_write":967,"num_write_kb":9811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"210'738","reported_seq":4425,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:57.277753+0000","last_change":"2024-09-13T09:10:57.277838+0000","last_active":"2024-09-13T09:10:57.277753+0000","last_peered":"2024-09-13T09:10:57.277753+0000","last_clean":"2024-09-13T09:10:57.277753+0000","last_became_active":"2024-09-13T09:10:36.918448+0000","last_became_peered":"2024-09-13T09:10:36.918448+0000","last_unstale":"2024-09-13T09:10:57.277753+0000","last_undegraded":"2024-09-13T09:10:57.277753+0000","last_fullsized":"2024-09-13T09:10:57.277753+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'738","last_scrub_stamp":"2024-09-13T09:10:57.277734+0000","last_deep_scrub":"210'738","last_deep_scrub_stamp":"2024-09-13T09:10:57.277734+0000","last_clean_scrub_stamp":"2024-09-13T09:10:57.277734+0000","objects_scrubbed":4,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:19.098979+0000","scrub_duration":0.0082513580000000003,"objects_trimmed":0,"snaptrim_duration":0.0021014559999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5305,"num_read_kb":24795,"num_write":695,"num_write_kb":16167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"210'733","reported_seq":19455,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:05.958813+0000","last_change":"2024-09-13T09:11:05.959077+0000","last_active":"2024-09-13T09:11:05.958813+0000","last_peered":"2024-09-13T09:11:05.958813+0000","last_clean":"2024-09-13T09:11:05.958813+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:11:05.958813+0000","last_undegraded":"2024-09-13T09:11:05.958813+0000","last_fullsized":"2024-09-13T09:11:05.958813+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'733","last_scrub_stamp":"2024-09-13T09:11:05.958759+0000","last_deep_scrub":"210'733","last_deep_scrub_stamp":"2024-09-13T09:11:05.958759+0000","last_clean_scrub_stamp":"2024-09-13T09:11:05.958759+0000","objects_scrubbed":8,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:23.769409+0000","scrub_duration":0.017881960999999998,"objects_trimmed":0,"snaptrim_duration":0.0012366219999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21128,"num_read_kb":34179,"num_write":635,"num_write_kb":10198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:59.489821+0000","last_change":"2024-09-13T09:10:59.490048+0000","last_active":"2024-09-13T09:10:59.489821+0000","last_peered":"2024-09-13T09:10:59.489821+0000","last_clean":"2024-09-13T09:10:59.489821+0000","last_became_active":"2024-09-13T09:10:33.349249+0000","last_became_peered":"2024-09-13T09:10:33.349249+0000","last_unstale":"2024-09-13T09:10:59.489821+0000","last_undegraded":"2024-09-13T09:10:59.489821+0000","last_fullsized":"2024-09-13T09:10:59.489821+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:10:59.489769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:10:59.489769+0000","last_clean_scrub_stamp":"2024-09-13T09:10:59.489769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:01.802332+0000","scrub_duration":0.002674531,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"210'709","reported_seq":6024,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:02.175990+0000","last_change":"2024-09-13T09:11:02.176223+0000","last_active":"2024-09-13T09:11:02.175990+0000","last_peered":"2024-09-13T09:11:02.175990+0000","last_clean":"2024-09-13T09:11:02.175990+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:11:02.175990+0000","last_undegraded":"2024-09-13T09:11:02.175990+0000","last_fullsized":"2024-09-13T09:11:02.175990+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'709","last_scrub_stamp":"2024-09-13T09:11:02.175935+0000","last_deep_scrub":"210'709","last_deep_scrub_stamp":"2024-09-13T09:11:02.175935+0000","last_clean_scrub_stamp":"2024-09-13T09:11:02.175935+0000","objects_scrubbed":9,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:23.880522+0000","scrub_duration":0.019129159,"objects_trimmed":0,"snaptrim_duration":0.0028872860000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6169,"num_read_kb":21942,"num_write":789,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":232,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:59.173464+0000","last_change":"2024-09-13T09:10:59.173686+0000","last_active":"2024-09-13T09:10:59.173464+0000","last_peered":"2024-09-13T09:10:59.173464+0000","last_clean":"2024-09-13T09:10:59.173464+0000","last_became_active":"2024-09-13T09:10:33.349192+0000","last_became_peered":"2024-09-13T09:10:33.349192+0000","last_unstale":"2024-09-13T09:10:59.173464+0000","last_undegraded":"2024-09-13T09:10:59.173464+0000","last_fullsized":"2024-09-13T09:10:59.173464+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:10:59.173413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:10:59.173413+0000","last_clean_scrub_stamp":"2024-09-13T09:10:59.173413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:28:07.754167+0000","scrub_duration":0.0033727399999999999,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"210'447","reported_seq":2569,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:12.030514+0000","last_change":"2024-09-13T09:11:12.030770+0000","last_active":"2024-09-13T09:11:12.030514+0000","last_peered":"2024-09-13T09:11:12.030514+0000","last_clean":"2024-09-13T09:11:12.030514+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:11:12.030514+0000","last_undegraded":"2024-09-13T09:11:12.030514+0000","last_fullsized":"2024-09-13T09:11:12.030514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'447","last_scrub_stamp":"2024-09-13T09:11:12.030454+0000","last_deep_scrub":"210'447","last_deep_scrub_stamp":"2024-09-13T09:11:12.030454+0000","last_clean_scrub_stamp":"2024-09-13T09:11:12.030454+0000","objects_scrubbed":14,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:41:50.134213+0000","scrub_duration":0.025469632999999998,"objects_trimmed":0,"snaptrim_duration":0.001256109,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2732,"num_read_kb":14647,"num_write":388,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":218,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:01.563398+0000","last_change":"2024-09-13T09:11:01.563398+0000","last_active":"2024-09-13T09:11:01.563398+0000","last_peered":"2024-09-13T09:11:01.563398+0000","last_clean":"2024-09-13T09:11:01.563398+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:11:01.563398+0000","last_undegraded":"2024-09-13T09:11:01.563398+0000","last_fullsized":"2024-09-13T09:11:01.563398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:01.563140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:01.563140+0000","last_clean_scrub_stamp":"2024-09-13T09:11:01.563140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:13:31.179427+0000","scrub_duration":0.0029921449999999999,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"210'630","reported_seq":6267,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.291387+0000","last_change":"2024-09-13T09:11:04.291622+0000","last_active":"2024-09-13T09:11:04.291387+0000","last_peered":"2024-09-13T09:11:04.291387+0000","last_clean":"2024-09-13T09:11:04.291387+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:11:04.291387+0000","last_undegraded":"2024-09-13T09:11:04.291387+0000","last_fullsized":"2024-09-13T09:11:04.291387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'630","last_scrub_stamp":"2024-09-13T09:11:04.291332+0000","last_deep_scrub":"210'630","last_deep_scrub_stamp":"2024-09-13T09:11:04.291332+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.291332+0000","objects_scrubbed":9,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.566099+0000","scrub_duration":0.017666608,"objects_trimmed":0,"snaptrim_duration":0.00043649599999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6810,"num_read_kb":26019,"num_write":814,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":219,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:01.891492+0000","last_change":"2024-09-13T09:11:01.891492+0000","last_active":"2024-09-13T09:11:01.891492+0000","last_peered":"2024-09-13T09:11:01.891492+0000","last_clean":"2024-09-13T09:11:01.891492+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:11:01.891492+0000","last_undegraded":"2024-09-13T09:11:01.891492+0000","last_fullsized":"2024-09-13T09:11:01.891492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:01.891225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:01.891225+0000","last_clean_scrub_stamp":"2024-09-13T09:11:01.891225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:41:05.422860+0000","scrub_duration":0.002855771,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"210'725","reported_seq":3907,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:03.507768+0000","last_change":"2024-09-13T09:11:03.507995+0000","last_active":"2024-09-13T09:11:03.507768+0000","last_peered":"2024-09-13T09:11:03.507768+0000","last_clean":"2024-09-13T09:11:03.507768+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:11:03.507768+0000","last_undegraded":"2024-09-13T09:11:03.507768+0000","last_fullsized":"2024-09-13T09:11:03.507768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'725","last_scrub_stamp":"2024-09-13T09:11:03.507714+0000","last_deep_scrub":"210'725","last_deep_scrub_stamp":"2024-09-13T09:11:03.507714+0000","last_clean_scrub_stamp":"2024-09-13T09:11:03.507714+0000","objects_scrubbed":8,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:38.375963+0000","scrub_duration":0.018454834,"objects_trimmed":0,"snaptrim_duration":0.0014721929999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4939,"num_read_kb":19757,"num_write":904,"num_write_kb":10484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"210'681","reported_seq":3565,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:58.282339+0000","last_change":"2024-09-13T09:10:58.282460+0000","last_active":"2024-09-13T09:10:58.282339+0000","last_peered":"2024-09-13T09:10:58.282339+0000","last_clean":"2024-09-13T09:10:58.282339+0000","last_became_active":"2024-09-13T09:10:36.917124+0000","last_became_peered":"2024-09-13T09:10:36.917124+0000","last_unstale":"2024-09-13T09:10:58.282339+0000","last_undegraded":"2024-09-13T09:10:58.282339+0000","last_fullsized":"2024-09-13T09:10:58.282339+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'681","last_scrub_stamp":"2024-09-13T09:10:58.282319+0000","last_deep_scrub":"210'681","last_deep_scrub_stamp":"2024-09-13T09:10:58.282319+0000","last_clean_scrub_stamp":"2024-09-13T09:10:58.282319+0000","objects_scrubbed":14,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:57.991183+0000","scrub_duration":0.021272119999999999,"objects_trimmed":0,"snaptrim_duration":0.0011757899999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3513,"num_read_kb":18416,"num_write":638,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":221,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:11.573384+0000","last_change":"2024-09-13T09:11:11.573384+0000","last_active":"2024-09-13T09:11:11.573384+0000","last_peered":"2024-09-13T09:11:11.573384+0000","last_clean":"2024-09-13T09:11:11.573384+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:11:11.573384+0000","last_undegraded":"2024-09-13T09:11:11.573384+0000","last_fullsized":"2024-09-13T09:11:11.573384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-13T09:11:11.573111+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-13T09:11:11.573111+0000","last_clean_scrub_stamp":"2024-09-13T09:11:11.573111+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:20.023637+0000","scrub_duration":0.0068124739999999998,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"210'523","reported_seq":3824,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:11.869828+0000","last_change":"2024-09-13T09:11:11.869828+0000","last_active":"2024-09-13T09:11:11.869828+0000","last_peered":"2024-09-13T09:11:11.869828+0000","last_clean":"2024-09-13T09:11:11.869828+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:11:11.869828+0000","last_undegraded":"2024-09-13T09:11:11.869828+0000","last_fullsized":"2024-09-13T09:11:11.869828+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'523","last_scrub_stamp":"2024-09-13T09:11:11.869514+0000","last_deep_scrub":"210'523","last_deep_scrub_stamp":"2024-09-13T09:11:11.869514+0000","last_clean_scrub_stamp":"2024-09-13T09:11:11.869514+0000","objects_scrubbed":19,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:46.036506+0000","scrub_duration":0.034765302999999997,"objects_trimmed":0,"snaptrim_duration":0.00095688200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3585,"num_read_kb":19320,"num_write":505,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:58.760700+0000","last_change":"2024-09-13T09:10:58.760935+0000","last_active":"2024-09-13T09:10:58.760700+0000","last_peered":"2024-09-13T09:10:58.760700+0000","last_clean":"2024-09-13T09:10:58.760700+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:58.760700+0000","last_undegraded":"2024-09-13T09:10:58.760700+0000","last_fullsized":"2024-09-13T09:10:58.760700+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:10:58.760647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:10:58.760647+0000","last_clean_scrub_stamp":"2024-09-13T09:10:58.760647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:17.592464+0000","scrub_duration":0.004023591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"210'432","reported_seq":2376,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:06.352185+0000","last_change":"2024-09-13T09:11:06.352417+0000","last_active":"2024-09-13T09:11:06.352185+0000","last_peered":"2024-09-13T09:11:06.352185+0000","last_clean":"2024-09-13T09:11:06.352185+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:11:06.352185+0000","last_undegraded":"2024-09-13T09:11:06.352185+0000","last_fullsized":"2024-09-13T09:11:06.352185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'432","last_scrub_stamp":"2024-09-13T09:11:06.352132+0000","last_deep_scrub":"210'432","last_deep_scrub_stamp":"2024-09-13T09:11:06.352132+0000","last_clean_scrub_stamp":"2024-09-13T09:11:06.352132+0000","objects_scrubbed":10,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:05.270131+0000","scrub_duration":0.016779835,"objects_trimmed":0,"snaptrim_duration":0.00074152199999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2071,"num_read_kb":14898,"num_write":412,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:02.558028+0000","last_change":"2024-09-13T09:11:02.558028+0000","last_active":"2024-09-13T09:11:02.558028+0000","last_peered":"2024-09-13T09:11:02.558028+0000","last_clean":"2024-09-13T09:11:02.558028+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:11:02.558028+0000","last_undegraded":"2024-09-13T09:11:02.558028+0000","last_fullsized":"2024-09-13T09:11:02.558028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:02.557758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:02.557758+0000","last_clean_scrub_stamp":"2024-09-13T09:11:02.557758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:06.344243+0000","scrub_duration":0.0032222179999999998,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"210'415","reported_seq":2601,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:09.080910+0000","last_change":"2024-09-13T09:11:09.081149+0000","last_active":"2024-09-13T09:11:09.080910+0000","last_peered":"2024-09-13T09:11:09.080910+0000","last_clean":"2024-09-13T09:11:09.080910+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:11:09.080910+0000","last_undegraded":"2024-09-13T09:11:09.080910+0000","last_fullsized":"2024-09-13T09:11:09.080910+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'415","last_scrub_stamp":"2024-09-13T09:11:09.080850+0000","last_deep_scrub":"210'415","last_deep_scrub_stamp":"2024-09-13T09:11:09.080850+0000","last_clean_scrub_stamp":"2024-09-13T09:11:09.080850+0000","objects_scrubbed":10,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:25:16.304447+0000","scrub_duration":0.019439997000000001,"objects_trimmed":0,"snaptrim_duration":0.001037313,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2540,"num_read_kb":12233,"num_write":423,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"210'2","reported_seq":220,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.597534+0000","last_change":"2024-09-13T09:11:04.597534+0000","last_active":"2024-09-13T09:11:04.597534+0000","last_peered":"2024-09-13T09:11:04.597534+0000","last_clean":"2024-09-13T09:11:04.597534+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:11:04.597534+0000","last_undegraded":"2024-09-13T09:11:04.597534+0000","last_fullsized":"2024-09-13T09:11:04.597534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'2","last_scrub_stamp":"2024-09-13T09:11:04.597248+0000","last_deep_scrub":"210'2","last_deep_scrub_stamp":"2024-09-13T09:11:04.597248+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.597248+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:56.082283+0000","scrub_duration":0.0055551189999999999,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"210'119","reported_seq":349,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.913895+0000","last_change":"2024-09-13T09:11:04.914128+0000","last_active":"2024-09-13T09:11:04.913895+0000","last_peered":"2024-09-13T09:11:04.913895+0000","last_clean":"2024-09-13T09:11:04.913895+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:11:04.913895+0000","last_undegraded":"2024-09-13T09:11:04.913895+0000","last_fullsized":"2024-09-13T09:11:04.913895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'119","last_scrub_stamp":"2024-09-13T09:11:04.913842+0000","last_deep_scrub":"210'119","last_deep_scrub_stamp":"2024-09-13T09:11:04.913842+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.913842+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:57.881164+0000","scrub_duration":0.0085814419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"210'893","reported_seq":6176,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.128452+0000","last_change":"2024-09-13T09:11:04.128803+0000","last_active":"2024-09-13T09:11:04.128452+0000","last_peered":"2024-09-13T09:11:04.128452+0000","last_clean":"2024-09-13T09:11:04.128452+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:11:04.128452+0000","last_undegraded":"2024-09-13T09:11:04.128452+0000","last_fullsized":"2024-09-13T09:11:04.128452+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'893","last_scrub_stamp":"2024-09-13T09:11:04.128368+0000","last_deep_scrub":"210'893","last_deep_scrub_stamp":"2024-09-13T09:11:04.128368+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.128368+0000","objects_scrubbed":12,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:53.087941+0000","scrub_duration":0.02779214,"objects_trimmed":0,"snaptrim_duration":0.0018329830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6854,"num_read_kb":23202,"num_write":1277,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"189'594","reported_seq":3646,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.468077+0000","last_change":"2024-09-13T09:10:27.288793+0000","last_active":"2024-09-13T09:10:47.468077+0000","last_peered":"2024-09-13T09:10:47.468077+0000","last_clean":"2024-09-13T09:10:47.468077+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:10:47.468077+0000","last_undegraded":"2024-09-13T09:10:47.468077+0000","last_fullsized":"2024-09-13T09:10:47.468077+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099698400000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4866,"num_read_kb":16848,"num_write":688,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"188'590","reported_seq":3111,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.555817+0000","last_change":"2024-09-13T09:10:27.326271+0000","last_active":"2024-09-13T09:10:47.555817+0000","last_peered":"2024-09-13T09:10:47.555817+0000","last_clean":"2024-09-13T09:10:47.555817+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:10:47.555817+0000","last_undegraded":"2024-09-13T09:10:47.555817+0000","last_fullsized":"2024-09-13T09:10:47.555817+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074287699999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4001,"num_read_kb":22630,"num_write":547,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"210'438","reported_seq":3900,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.475858+0000","last_change":"2024-09-13T09:11:04.476097+0000","last_active":"2024-09-13T09:11:04.475858+0000","last_peered":"2024-09-13T09:11:04.475858+0000","last_clean":"2024-09-13T09:11:04.475858+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:11:04.475858+0000","last_undegraded":"2024-09-13T09:11:04.475858+0000","last_fullsized":"2024-09-13T09:11:04.475858+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'438","last_scrub_stamp":"2024-09-13T09:11:04.475792+0000","last_deep_scrub":"210'438","last_deep_scrub_stamp":"2024-09-13T09:11:04.475792+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.475792+0000","objects_scrubbed":10,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:09.660309+0000","scrub_duration":0.018211029,"objects_trimmed":0,"snaptrim_duration":0.0013662279999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3594,"num_read_kb":33920,"num_write":421,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"210'372","reported_seq":17082,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:56.293946+0000","last_change":"2024-09-13T09:10:56.294059+0000","last_active":"2024-09-13T09:10:56.293946+0000","last_peered":"2024-09-13T09:10:56.293946+0000","last_clean":"2024-09-13T09:10:56.293946+0000","last_became_active":"2024-09-13T09:10:35.919376+0000","last_became_peered":"2024-09-13T09:10:35.919376+0000","last_unstale":"2024-09-13T09:10:56.293946+0000","last_undegraded":"2024-09-13T09:10:56.293946+0000","last_fullsized":"2024-09-13T09:10:56.293946+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'372","last_scrub_stamp":"2024-09-13T09:10:56.293925+0000","last_deep_scrub":"210'372","last_deep_scrub_stamp":"2024-09-13T09:10:56.293925+0000","last_clean_scrub_stamp":"2024-09-13T09:10:56.293925+0000","objects_scrubbed":12,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:32.732009+0000","scrub_duration":0.014972901,"objects_trimmed":0,"snaptrim_duration":0.00088103200000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":16596,"num_read_kb":27933,"num_write":361,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"190'410","reported_seq":3138,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.468417+0000","last_change":"2024-09-13T09:10:27.289763+0000","last_active":"2024-09-13T09:10:47.468417+0000","last_peered":"2024-09-13T09:10:47.468417+0000","last_clean":"2024-09-13T09:10:47.468417+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:10:47.468417+0000","last_undegraded":"2024-09-13T09:10:47.468417+0000","last_fullsized":"2024-09-13T09:10:47.468417+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001690069,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3095,"num_read_kb":9820,"num_write":360,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"210'893","reported_seq":7393,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:57.491271+0000","last_change":"2024-09-13T09:10:57.491494+0000","last_active":"2024-09-13T09:10:57.491271+0000","last_peered":"2024-09-13T09:10:57.491271+0000","last_clean":"2024-09-13T09:10:57.491271+0000","last_became_active":"2024-09-13T09:10:35.914456+0000","last_became_peered":"2024-09-13T09:10:35.914456+0000","last_unstale":"2024-09-13T09:10:57.491271+0000","last_undegraded":"2024-09-13T09:10:57.491271+0000","last_fullsized":"2024-09-13T09:10:57.491271+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'893","last_scrub_stamp":"2024-09-13T09:10:57.491220+0000","last_deep_scrub":"210'893","last_deep_scrub_stamp":"2024-09-13T09:10:57.491220+0000","last_clean_scrub_stamp":"2024-09-13T09:10:57.491220+0000","objects_scrubbed":9,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:57:11.835289+0000","scrub_duration":0.018083443000000001,"objects_trimmed":0,"snaptrim_duration":0.001663692,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9292,"num_read_kb":23776,"num_write":1223,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"210'490","reported_seq":5203,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:02.258332+0000","last_change":"2024-09-13T09:11:02.258568+0000","last_active":"2024-09-13T09:11:02.258332+0000","last_peered":"2024-09-13T09:11:02.258332+0000","last_clean":"2024-09-13T09:11:02.258332+0000","last_became_active":"2024-09-13T09:10:35.915413+0000","last_became_peered":"2024-09-13T09:10:35.915413+0000","last_unstale":"2024-09-13T09:11:02.258332+0000","last_undegraded":"2024-09-13T09:11:02.258332+0000","last_fullsized":"2024-09-13T09:11:02.258332+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'490","last_scrub_stamp":"2024-09-13T09:11:02.258279+0000","last_deep_scrub":"210'490","last_deep_scrub_stamp":"2024-09-13T09:11:02.258279+0000","last_clean_scrub_stamp":"2024-09-13T09:11:02.258279+0000","objects_scrubbed":13,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:58:47.729702+0000","scrub_duration":0.025685860000000001,"objects_trimmed":0,"snaptrim_duration":0.001550994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5016,"num_read_kb":17262,"num_write":425,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"210'533","reported_seq":2934,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:02.921486+0000","last_change":"2024-09-13T09:11:02.921712+0000","last_active":"2024-09-13T09:11:02.921486+0000","last_peered":"2024-09-13T09:11:02.921486+0000","last_clean":"2024-09-13T09:11:02.921486+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:11:02.921486+0000","last_undegraded":"2024-09-13T09:11:02.921486+0000","last_fullsized":"2024-09-13T09:11:02.921486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'533","last_scrub_stamp":"2024-09-13T09:11:02.921433+0000","last_deep_scrub":"210'533","last_deep_scrub_stamp":"2024-09-13T09:11:02.921433+0000","last_clean_scrub_stamp":"2024-09-13T09:11:02.921433+0000","objects_scrubbed":15,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:53:22.794757+0000","scrub_duration":0.025615438000000001,"objects_trimmed":0,"snaptrim_duration":0.0011980339999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2903,"num_read_kb":17169,"num_write":491,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"190'1187","reported_seq":4412,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.468167+0000","last_change":"2024-09-13T09:10:27.289049+0000","last_active":"2024-09-13T09:10:47.468167+0000","last_peered":"2024-09-13T09:10:47.468167+0000","last_clean":"2024-09-13T09:10:47.468167+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:10:47.468167+0000","last_undegraded":"2024-09-13T09:10:47.468167+0000","last_fullsized":"2024-09-13T09:10:47.468167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001456372,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7816,"num_read_kb":12367,"num_write":1664,"num_write_kb":5387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"210'725","reported_seq":4634,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:58.292323+0000","last_change":"2024-09-13T09:10:58.292581+0000","last_active":"2024-09-13T09:10:58.292323+0000","last_peered":"2024-09-13T09:10:58.292323+0000","last_clean":"2024-09-13T09:10:58.292323+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:58.292323+0000","last_undegraded":"2024-09-13T09:10:58.292323+0000","last_fullsized":"2024-09-13T09:10:58.292323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'725","last_scrub_stamp":"2024-09-13T09:10:58.292265+0000","last_deep_scrub":"210'725","last_deep_scrub_stamp":"2024-09-13T09:10:58.292265+0000","last_clean_scrub_stamp":"2024-09-13T09:10:58.292265+0000","objects_scrubbed":7,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:09.243645+0000","scrub_duration":0.01780489,"objects_trimmed":0,"snaptrim_duration":0.0012075230000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5826,"num_read_kb":14063,"num_write":971,"num_write_kb":11379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"187'588","reported_seq":4154,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:47.467985+0000","last_change":"2024-09-13T09:10:31.329333+0000","last_active":"2024-09-13T09:10:47.467985+0000","last_peered":"2024-09-13T09:10:47.467985+0000","last_clean":"2024-09-13T09:10:47.467985+0000","last_became_active":"2024-09-13T09:10:31.325692+0000","last_became_peered":"2024-09-13T09:10:31.325692+0000","last_unstale":"2024-09-13T09:10:47.467985+0000","last_undegraded":"2024-09-13T09:10:47.467985+0000","last_fullsized":"2024-09-13T09:10:47.467985+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:07:13.155504+0000","last_clean_scrub_stamp":"2024-09-13T09:07:13.155504+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015796709999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3375,"num_read_kb":13502,"num_write":606,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":336,"num_read":210850,"num_read_kb":690514,"num_write":25042,"num_write_kb":338108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3692,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":21685,"ondisk_log_size":21685,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080589,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":2472,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]}]},{"osd":3,"up_from":15,"seq":64424509496,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69684,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302156,"statfs":{"total":96636764160,"available":96565407744,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.23100000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.91600000000000004,"5min":1.361,"15min":1.361},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63600000000000001,"15min":0.64700000000000002},"min":{"1min":0.34000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.97099999999999997,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.66900000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":5.7080000000000002,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63200000000000001,"15min":0.64800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.996,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.70999999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.35699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":1.004,"5min":1.444,"15min":1.444},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.68700000000000006,"15min":0.60299999999999998},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.8109999999999999,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":0.89000000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84099999999999997,"15min":0.84899999999999998},"min":{"1min":0.52000000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":0.91400000000000003,"15min":0.878},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.8449999999999998,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.84499999999999997,"15min":0.83099999999999996},"min":{"1min":0.47699999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.93999999999999995}]},{"osd":6,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.96899999999999997,"15min":0.92200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":5.6260000000000003,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":0.997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.92000000000000004,"15min":0.91900000000000004},"min":{"1min":0.71299999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.375,"5min":1.379,"15min":1.379},"last":0.97199999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.91000000000000003,"15min":0.90400000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.3819999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.95999999999999996,"15min":0.96399999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.381,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.113}]}]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69680,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302160,"statfs":{"total":96636764160,"available":96565411840,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.63700000000000001,"15min":0.68799999999999994},"min":{"1min":0.246,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91300000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.70299999999999996,"15min":0.73899999999999999},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84699999999999998,"5min":1.353,"15min":1.353},"last":0.93300000000000005}]},{"osd":1,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.61699999999999999,"15min":0.64900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.80800000000000005,"5min":1.484,"15min":1.484},"last":1.153},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.73799999999999999,"15min":0.77600000000000002},"min":{"1min":0.33600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":1.0109999999999999,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73199999999999998,"15min":0.79300000000000004},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.94999999999999996,"5min":1.681,"15min":1.681},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.80400000000000005,"15min":0.88800000000000001},"min":{"1min":0.41599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":1.179,"5min":1.401,"15min":1.401},"last":0.80300000000000005}]},{"osd":4,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.91600000000000004,"15min":1.008},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1759999999999999,"5min":1.998,"15min":1.998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.89700000000000002,"15min":0.94199999999999995},"min":{"1min":0.44900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1830000000000001,"5min":1.651,"15min":1.651},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.93200000000000005,"15min":0.97199999999999998},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.127,"5min":1.506,"15min":1.506},"last":1.026},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.95799999999999996,"15min":1.03},"min":{"1min":0.438,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.254,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.05}]},{"osd":6,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.006,"15min":1.0649999999999999},"min":{"1min":0.53900000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.494,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.002},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0429999999999999,"15min":1.109},"min":{"1min":0.59499999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2829999999999999,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.129}]},{"osd":7,"last update":"Fri Sep 13 09:09:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":1.109,"15min":1.1339999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.1869999999999998,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.099},{"interface":"front","average":{"1min":0.91200000000000003,"5min":1.0780000000000001,"15min":1.1060000000000001},"min":{"1min":0.52700000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2889999999999999,"5min":1.754,"15min":1.754},"last":1.077}]}]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56312,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315528,"statfs":{"total":96636764160,"available":96579100672,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.26700000000000002,"5min":0.221,"15min":0.221},"max":{"1min":1.3049999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.63900000000000001},"min":{"1min":0.33000000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.65400000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.26900000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.139},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.32400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.82699999999999996}]},{"osd":3,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.69099999999999995,"15min":0.72399999999999998},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.86499999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.86899999999999999,"15min":0.85599999999999998},"min":{"1min":0.55600000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.7930000000000001,"5min":3.7930000000000001,"15min":3.7930000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.66700000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.038}]},{"osd":5,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.95799999999999996,"15min":0.98399999999999999},"min":{"1min":0.51800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.96099999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.351,"5min":3.351,"15min":3.351},"last":1.1839999999999999}]},{"osd":6,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.93500000000000005,"15min":0.95199999999999996},"min":{"1min":0.54000000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.75,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.014,"15min":1.0509999999999999},"min":{"1min":0.61799999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8899999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.004}]},{"osd":7,"last update":"Fri Sep 13 09:08:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.0629999999999999,"15min":1.089},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3959999999999999,"5min":2.3929999999999998,"15min":2.3929999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.57699999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.306,"5min":1.679,"15min":1.679},"last":1.119}]}]},{"osd":0,"up_from":15,"seq":64424509497,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.378,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.79000000000000004}]},{"osd":2,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64200000000000002,"15min":0.69399999999999995},"min":{"1min":0.33800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96199999999999997,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.751,"15min":0.85299999999999998},"min":{"1min":0.28199999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.3140000000000001,"5min":3.855,"15min":3.855},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.80400000000000005},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":1.107,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.91300000000000003},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4390000000000001,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.84799999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.50900000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3660000000000001,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.872,"15min":0.93100000000000005},"min":{"1min":0.61199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.139,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.93999999999999995,"15min":0.97399999999999998},"min":{"1min":0.70599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.583,"5min":2.532,"15min":2.532},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.056,"15min":1.121},"min":{"1min":0.70999999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.698,"5min":2.181,"15min":2.181},"last":1.046}]},{"osd":6,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0169999999999999,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.98099999999999998,"15min":1.0609999999999999},"min":{"1min":0.54700000000000004,"5min":0.44,"15min":0.44},"max":{"1min":1.347,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.129}]},{"osd":7,"last update":"Fri Sep 13 09:08:57 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":1.044,"15min":1.0640000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.528,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.0840000000000001,"15min":1.1220000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4890000000000001,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":1.1870000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542200,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37684,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334156,"statfs":{"total":96636764160,"available":96598175744,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79800000000000004,"15min":0.82199999999999995},"min":{"1min":0.54400000000000004,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.3089999999999999,"5min":1.552,"15min":1.552},"last":0.753},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.80900000000000005,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1599999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.92100000000000004}]},{"osd":1,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.83299999999999996,"15min":0.80100000000000005},"min":{"1min":0.50600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2929999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.89400000000000002,"15min":0.91600000000000004},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4159999999999999,"5min":2.706,"15min":2.706},"last":0.97299999999999998}]},{"osd":2,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.325,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.96299999999999997,"15min":0.93799999999999994},"min":{"1min":0.61099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.22,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.145,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.92400000000000004},"min":{"1min":0.45100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.26}]},{"osd":4,"last update":"Fri Sep 13 09:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.96499999999999997,"15min":0.98899999999999999},"min":{"1min":0.53700000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2949999999999999,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.107},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91000000000000003,"15min":0.95999999999999996},"min":{"1min":0.39200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.7,"15min":1.7},"last":1.151}]},{"osd":5,"last update":"Fri Sep 13 09:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.054,"15min":1.083},"min":{"1min":0.52800000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.6339999999999999,"5min":1.776,"15min":1.776},"last":1.198},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.581,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.1799999999999999}]},{"osd":7,"last update":"Fri Sep 13 09:09:02 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1160000000000001,"15min":1.1220000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6759999999999999,"5min":1.792,"15min":1.792},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0860000000000001,"15min":1.1060000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.516,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.046}]}]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47536,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324304,"statfs":{"total":96636764160,"available":96588087296,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80200000000000005,"15min":0.83499999999999996},"min":{"1min":0.31,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.1379999999999999,"5min":2.649,"15min":2.649},"last":1.1379999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.76500000000000001,"15min":0.74299999999999999},"min":{"1min":0.35499999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.1100000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.135,"5min":1.3,"15min":1.3},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Sep 13 09:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90000000000000002,"15min":0.83799999999999997},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2769999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":1.018},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.93100000000000005,"15min":0.92300000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.206,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Sep 13 09:11:11 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.94999999999999996,"15min":0.879},"min":{"1min":0.48799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.409,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.98199999999999998,"15min":0.97899999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.452,"5min":1.573,"15min":1.573},"last":1.0760000000000001}]},{"osd":5,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91900000000000004,"15min":0.86399999999999999},"min":{"1min":0.377,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.6830000000000001,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0009999999999999,"15min":0.90400000000000003},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.756,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.40200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.522,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0449999999999999,"15min":1.04},"min":{"1min":0.442,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.792,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.2250000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:06 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.008,"15min":1.0069999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.9020000000000001,"5min":2.9020000000000001,"15min":2.9020000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.147,"5min":0.97799999999999998,"15min":0.94999999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.8439999999999999,"5min":3.8439999999999999,"15min":3.8439999999999999},"last":1.1659999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48120,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323720,"statfs":{"total":96636764160,"available":96587489280,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79600000000000004,"15min":0.79700000000000004},"min":{"1min":0.48199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1439999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.042},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.97799999999999998,"15min":1.1319999999999999},"min":{"1min":0.622,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.194,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.877}]},{"osd":1,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.78100000000000003,"15min":0.82899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.28,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.276},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2809999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.95399999999999996,"15min":1.1080000000000001},"min":{"1min":0.49199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.155,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":1.2370000000000001},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.68600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0940000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":1.006}]},{"osd":3,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.01,"15min":1.173},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1679999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93400000000000005,"15min":0.96599999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2609999999999999,"5min":1.585,"15min":1.585},"last":1.1279999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.97799999999999998,"15min":1.018},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.321,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.147,"15min":1.345},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.417,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.0800000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.98399999999999999,"15min":0.98599999999999999},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.391,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.0529999999999999,"15min":1.0820000000000001},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.518,"5min":1.899,"15min":1.899},"last":1.2130000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:09:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.3740000000000001,"15min":1.4239999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.4450000000000001,"5min":8.1890000000000001,"15min":8.1890000000000001},"last":1.165},{"interface":"front","average":{"1min":1.101,"5min":1.169,"15min":1.1799999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.675,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.3320000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:11:19.132 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:11:19.132 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:11:19.133 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:11:19.133 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:11:19.133 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:11:19.133 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:11:19.133 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T09:07:13.155504+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=7, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=9, tm_min=10, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T09:11:19.133 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T09:11:39.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T09:11:39.458 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-13T09:11:39.459 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-13T09:11:39.477 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-13T09:11:39.128354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":336,"num_read":210966,"num_read_kb":690614,"num_write":25203,"num_write_kb":340526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3694,"num_bytes_recovered":209675492,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21884,"ondisk_log_size":21884,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":398040,"kb_used_data":20952,"kb_used_omap":30,"kb_used_meta":376993,"kb_avail":754576680,"statfs":{"total":773094113280,"available":772686520320,"internally_reserved":0,"allocated":21454848,"data_stored":10992532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31401,"internal_metadata":386041175},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003871"},"pg_stats":[{"pgid":"3.1f","version":"210'582","reported_seq":5761,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:17.825897+0000","last_change":"2024-09-13T09:11:17.825897+0000","last_active":"2024-09-13T09:11:17.825897+0000","last_peered":"2024-09-13T09:11:17.825897+0000","last_clean":"2024-09-13T09:11:17.825897+0000","last_became_active":"2024-09-13T09:07:53.862599+0000","last_became_peered":"2024-09-13T09:07:53.862599+0000","last_unstale":"2024-09-13T09:11:17.825897+0000","last_undegraded":"2024-09-13T09:11:17.825897+0000","last_fullsized":"2024-09-13T09:11:17.825897+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'582","last_scrub_stamp":"2024-09-13T09:11:17.825604+0000","last_deep_scrub":"210'582","last_deep_scrub_stamp":"2024-09-13T09:11:17.825604+0000","last_clean_scrub_stamp":"2024-09-13T09:11:17.825604+0000","objects_scrubbed":13,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:43:35.931565+0000","scrub_duration":0.021020661999999999,"objects_trimmed":0,"snaptrim_duration":0.0016218700000000001,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5971,"num_read_kb":20218,"num_write":677,"num_write_kb":6245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":489655,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"210'1237","reported_seq":5167,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:10.834228+0000","last_change":"2024-09-13T09:11:10.834228+0000","last_active":"2024-09-13T09:11:10.834228+0000","last_peered":"2024-09-13T09:11:10.834228+0000","last_clean":"2024-09-13T09:11:10.834228+0000","last_became_active":"2024-09-13T09:07:13.857041+0000","last_became_peered":"2024-09-13T09:07:13.857041+0000","last_unstale":"2024-09-13T09:11:10.834228+0000","last_undegraded":"2024-09-13T09:11:10.834228+0000","last_fullsized":"2024-09-13T09:11:10.834228+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'1237","last_scrub_stamp":"2024-09-13T09:11:10.833981+0000","last_deep_scrub":"210'1237","last_deep_scrub_stamp":"2024-09-13T09:11:10.833981+0000","last_clean_scrub_stamp":"2024-09-13T09:11:10.833981+0000","objects_scrubbed":11,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:27:13.914436+0000","scrub_duration":0.021379769,"objects_trimmed":0,"snaptrim_duration":0.00082279900000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":9326,"num_read_kb":19231,"num_write":1571,"num_write_kb":6032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"210'1425","reported_seq":5720,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:27.627312+0000","last_change":"2024-09-13T09:11:27.627562+0000","last_active":"2024-09-13T09:11:27.627312+0000","last_peered":"2024-09-13T09:11:27.627312+0000","last_clean":"2024-09-13T09:11:27.627312+0000","last_became_active":"2024-09-13T09:10:37.936189+0000","last_became_peered":"2024-09-13T09:10:37.936189+0000","last_unstale":"2024-09-13T09:11:27.627312+0000","last_undegraded":"2024-09-13T09:11:27.627312+0000","last_fullsized":"2024-09-13T09:11:27.627312+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'1425","last_scrub_stamp":"2024-09-13T09:11:27.627257+0000","last_deep_scrub":"210'1425","last_deep_scrub_stamp":"2024-09-13T09:11:27.627257+0000","last_clean_scrub_stamp":"2024-09-13T09:11:27.627257+0000","objects_scrubbed":16,"log_size":1425,"log_dups_size":0,"ondisk_log_size":1425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:23:33.200287+0000","scrub_duration":0.039007411999999998,"objects_trimmed":0,"snaptrim_duration":0.0013860960000000001,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":8733,"num_read_kb":34149,"num_write":2234,"num_write_kb":17361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":7412523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"210'742","reported_seq":4407,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:00.907957+0000","last_change":"2024-09-13T09:11:00.908182+0000","last_active":"2024-09-13T09:11:00.907957+0000","last_peered":"2024-09-13T09:11:00.907957+0000","last_clean":"2024-09-13T09:11:00.907957+0000","last_became_active":"2024-09-13T09:10:36.913240+0000","last_became_peered":"2024-09-13T09:10:36.913240+0000","last_unstale":"2024-09-13T09:11:00.907957+0000","last_undegraded":"2024-09-13T09:11:00.907957+0000","last_fullsized":"2024-09-13T09:11:00.907957+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'742","last_scrub_stamp":"2024-09-13T09:11:00.907901+0000","last_deep_scrub":"210'742","last_deep_scrub_stamp":"2024-09-13T09:11:00.907901+0000","last_clean_scrub_stamp":"2024-09-13T09:11:00.907901+0000","objects_scrubbed":10,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:41.042092+0000","scrub_duration":0.022088087999999999,"objects_trimmed":0,"snaptrim_duration":0.0015845119999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":5745,"num_read_kb":35631,"num_write":972,"num_write_kb":15639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":9785257,"num_keys_recovered":116,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"210'738","reported_seq":22484,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:03.159232+0000","last_change":"2024-09-13T09:11:03.159453+0000","last_active":"2024-09-13T09:11:03.159232+0000","last_peered":"2024-09-13T09:11:03.159232+0000","last_clean":"2024-09-13T09:11:03.159232+0000","last_became_active":"2024-09-13T09:10:36.914093+0000","last_became_peered":"2024-09-13T09:10:36.914093+0000","last_unstale":"2024-09-13T09:11:03.159232+0000","last_undegraded":"2024-09-13T09:11:03.159232+0000","last_fullsized":"2024-09-13T09:11:03.159232+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'738","last_scrub_stamp":"2024-09-13T09:11:03.159180+0000","last_deep_scrub":"210'738","last_deep_scrub_stamp":"2024-09-13T09:11:03.159180+0000","last_clean_scrub_stamp":"2024-09-13T09:11:03.159180+0000","objects_scrubbed":8,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:37.381128+0000","scrub_duration":0.015880679000000002,"objects_trimmed":0,"snaptrim_duration":0.0023266060000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":23676,"num_read_kb":48180,"num_write":787,"num_write_kb":18573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":7792645,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"210'718","reported_seq":6562,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:01.943284+0000","last_change":"2024-09-13T09:11:01.943676+0000","last_active":"2024-09-13T09:11:01.943284+0000","last_peered":"2024-09-13T09:11:01.943284+0000","last_clean":"2024-09-13T09:11:01.943284+0000","last_became_active":"2024-09-13T09:08:09.862558+0000","last_became_peered":"2024-09-13T09:08:09.862558+0000","last_unstale":"2024-09-13T09:11:01.943284+0000","last_undegraded":"2024-09-13T09:11:01.943284+0000","last_fullsized":"2024-09-13T09:11:01.943284+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'718","last_scrub_stamp":"2024-09-13T09:11:01.943177+0000","last_deep_scrub":"210'718","last_deep_scrub_stamp":"2024-09-13T09:11:01.943177+0000","last_clean_scrub_stamp":"2024-09-13T09:11:01.943177+0000","objects_scrubbed":14,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:04:22.134328+0000","scrub_duration":0.026873905999999999,"objects_trimmed":0,"snaptrim_duration":0.00093814799999999998,"stat_sum":{"num_bytes":19,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":8455,"num_read_kb":25867,"num_write":848,"num_write_kb":19677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":48012029,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"210'401","reported_seq":2142,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:10.034913+0000","last_change":"2024-09-13T09:11:10.035158+0000","last_active":"2024-09-13T09:11:10.034913+0000","last_peered":"2024-09-13T09:11:10.034913+0000","last_clean":"2024-09-13T09:11:10.034913+0000","last_became_active":"2024-09-13T09:07:50.837001+0000","last_became_peered":"2024-09-13T09:07:50.837001+0000","last_unstale":"2024-09-13T09:11:10.034913+0000","last_undegraded":"2024-09-13T09:11:10.034913+0000","last_fullsized":"2024-09-13T09:11:10.034913+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'401","last_scrub_stamp":"2024-09-13T09:11:10.034858+0000","last_deep_scrub":"210'401","last_deep_scrub_stamp":"2024-09-13T09:11:10.034858+0000","last_clean_scrub_stamp":"2024-09-13T09:11:10.034858+0000","objects_scrubbed":14,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:06.863823+0000","scrub_duration":0.021874984,"objects_trimmed":0,"snaptrim_duration":0.002340506,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1827,"num_read_kb":8140,"num_write":348,"num_write_kb":7304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":524815,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"210'725","reported_seq":3600,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:06.902907+0000","last_change":"2024-09-13T09:11:06.903138+0000","last_active":"2024-09-13T09:11:06.902907+0000","last_peered":"2024-09-13T09:11:06.902907+0000","last_clean":"2024-09-13T09:11:06.902907+0000","last_became_active":"2024-09-13T09:07:13.864785+0000","last_became_peered":"2024-09-13T09:07:13.864785+0000","last_unstale":"2024-09-13T09:11:06.902907+0000","last_undegraded":"2024-09-13T09:11:06.902907+0000","last_fullsized":"2024-09-13T09:11:06.902907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'725","last_scrub_stamp":"2024-09-13T09:11:06.902854+0000","last_deep_scrub":"210'725","last_deep_scrub_stamp":"2024-09-13T09:11:06.902854+0000","last_clean_scrub_stamp":"2024-09-13T09:11:06.902854+0000","objects_scrubbed":11,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:02:48.704124+0000","scrub_duration":0.021786298999999999,"objects_trimmed":0,"snaptrim_duration":0.0016018569999999999,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4805,"num_read_kb":14792,"num_write":763,"num_write_kb":10450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"210'674","reported_seq":5984,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:07.858713+0000","last_change":"2024-09-13T09:11:07.859015+0000","last_active":"2024-09-13T09:11:07.858713+0000","last_peered":"2024-09-13T09:11:07.858713+0000","last_clean":"2024-09-13T09:11:07.858713+0000","last_became_active":"2024-09-13T09:07:13.861393+0000","last_became_peered":"2024-09-13T09:07:13.861393+0000","last_unstale":"2024-09-13T09:11:07.858713+0000","last_undegraded":"2024-09-13T09:11:07.858713+0000","last_fullsized":"2024-09-13T09:11:07.858713+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'674","last_scrub_stamp":"2024-09-13T09:11:07.858660+0000","last_deep_scrub":"210'674","last_deep_scrub_stamp":"2024-09-13T09:11:07.858660+0000","last_clean_scrub_stamp":"2024-09-13T09:11:07.858660+0000","objects_scrubbed":6,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:50.834925+0000","scrub_duration":0.015685364,"objects_trimmed":0,"snaptrim_duration":0.0012689800000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":7350,"num_read_kb":31391,"num_write":638,"num_write_kb":18271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"210'726","reported_seq":2149,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:58.509695+0000","last_change":"2024-09-13T09:10:58.509825+0000","last_active":"2024-09-13T09:10:58.509695+0000","last_peered":"2024-09-13T09:10:58.509695+0000","last_clean":"2024-09-13T09:10:58.509695+0000","last_became_active":"2024-09-13T09:07:13.860041+0000","last_became_peered":"2024-09-13T09:07:13.860041+0000","last_unstale":"2024-09-13T09:10:58.509695+0000","last_undegraded":"2024-09-13T09:10:58.509695+0000","last_fullsized":"2024-09-13T09:10:58.509695+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'726","last_scrub_stamp":"2024-09-13T09:10:58.509653+0000","last_deep_scrub":"210'726","last_deep_scrub_stamp":"2024-09-13T09:10:58.509653+0000","last_clean_scrub_stamp":"2024-09-13T09:10:58.509653+0000","objects_scrubbed":8,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:14:02.988273+0000","scrub_duration":0.0086180700000000002,"objects_trimmed":0,"snaptrim_duration":0.0020123239999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2936,"num_read_kb":14217,"num_write":967,"num_write_kb":9811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"210'738","reported_seq":4425,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:57.277753+0000","last_change":"2024-09-13T09:10:57.277838+0000","last_active":"2024-09-13T09:10:57.277753+0000","last_peered":"2024-09-13T09:10:57.277753+0000","last_clean":"2024-09-13T09:10:57.277753+0000","last_became_active":"2024-09-13T09:10:36.918448+0000","last_became_peered":"2024-09-13T09:10:36.918448+0000","last_unstale":"2024-09-13T09:10:57.277753+0000","last_undegraded":"2024-09-13T09:10:57.277753+0000","last_fullsized":"2024-09-13T09:10:57.277753+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'738","last_scrub_stamp":"2024-09-13T09:10:57.277734+0000","last_deep_scrub":"210'738","last_deep_scrub_stamp":"2024-09-13T09:10:57.277734+0000","last_clean_scrub_stamp":"2024-09-13T09:10:57.277734+0000","objects_scrubbed":4,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:19.098979+0000","scrub_duration":0.0082513580000000003,"objects_trimmed":0,"snaptrim_duration":0.0021014559999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":5305,"num_read_kb":24795,"num_write":695,"num_write_kb":16167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":213,"num_bytes_recovered":5681840,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"210'733","reported_seq":19455,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:05.958813+0000","last_change":"2024-09-13T09:11:05.959077+0000","last_active":"2024-09-13T09:11:05.958813+0000","last_peered":"2024-09-13T09:11:05.958813+0000","last_clean":"2024-09-13T09:11:05.958813+0000","last_became_active":"2024-09-13T09:09:24.172361+0000","last_became_peered":"2024-09-13T09:09:24.172361+0000","last_unstale":"2024-09-13T09:11:05.958813+0000","last_undegraded":"2024-09-13T09:11:05.958813+0000","last_fullsized":"2024-09-13T09:11:05.958813+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'733","last_scrub_stamp":"2024-09-13T09:11:05.958759+0000","last_deep_scrub":"210'733","last_deep_scrub_stamp":"2024-09-13T09:11:05.958759+0000","last_clean_scrub_stamp":"2024-09-13T09:11:05.958759+0000","objects_scrubbed":8,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:55:23.769409+0000","scrub_duration":0.017881960999999998,"objects_trimmed":0,"snaptrim_duration":0.0012366219999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":21128,"num_read_kb":34179,"num_write":635,"num_write_kb":10198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":7395735,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":234,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:59.490048+0000","last_change":"2024-09-13T09:10:59.490048+0000","last_active":"2024-09-13T09:10:59.490048+0000","last_peered":"2024-09-13T09:10:59.490048+0000","last_clean":"2024-09-13T09:10:59.490048+0000","last_became_active":"2024-09-13T09:10:33.349249+0000","last_became_peered":"2024-09-13T09:10:33.349249+0000","last_unstale":"2024-09-13T09:10:59.490048+0000","last_undegraded":"2024-09-13T09:10:59.490048+0000","last_fullsized":"2024-09-13T09:10:59.490048+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:10:59.489769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:10:59.489769+0000","last_clean_scrub_stamp":"2024-09-13T09:10:59.489769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:38:01.802332+0000","scrub_duration":0.002674531,"objects_trimmed":0,"snaptrim_duration":0.0026484019999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"210'709","reported_seq":6025,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:02.176223+0000","last_change":"2024-09-13T09:11:02.176223+0000","last_active":"2024-09-13T09:11:02.176223+0000","last_peered":"2024-09-13T09:11:02.176223+0000","last_clean":"2024-09-13T09:11:02.176223+0000","last_became_active":"2024-09-13T09:10:19.203706+0000","last_became_peered":"2024-09-13T09:10:19.203706+0000","last_unstale":"2024-09-13T09:11:02.176223+0000","last_undegraded":"2024-09-13T09:11:02.176223+0000","last_fullsized":"2024-09-13T09:11:02.176223+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'709","last_scrub_stamp":"2024-09-13T09:11:02.175935+0000","last_deep_scrub":"210'709","last_deep_scrub_stamp":"2024-09-13T09:11:02.175935+0000","last_clean_scrub_stamp":"2024-09-13T09:11:02.175935+0000","objects_scrubbed":9,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:51:23.880522+0000","scrub_duration":0.019129159,"objects_trimmed":0,"snaptrim_duration":0.0028872860000000002,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6169,"num_read_kb":21942,"num_write":789,"num_write_kb":6970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":7943911,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":233,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:59.173686+0000","last_change":"2024-09-13T09:10:59.173686+0000","last_active":"2024-09-13T09:10:59.173686+0000","last_peered":"2024-09-13T09:10:59.173686+0000","last_clean":"2024-09-13T09:10:59.173686+0000","last_became_active":"2024-09-13T09:10:33.349192+0000","last_became_peered":"2024-09-13T09:10:33.349192+0000","last_unstale":"2024-09-13T09:10:59.173686+0000","last_undegraded":"2024-09-13T09:10:59.173686+0000","last_fullsized":"2024-09-13T09:10:59.173686+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:10:59.173413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:10:59.173413+0000","last_clean_scrub_stamp":"2024-09-13T09:10:59.173413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:28:07.754167+0000","scrub_duration":0.0033727399999999999,"objects_trimmed":0,"snaptrim_duration":0.00085980900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"210'447","reported_seq":2570,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:12.030770+0000","last_change":"2024-09-13T09:11:12.030770+0000","last_active":"2024-09-13T09:11:12.030770+0000","last_peered":"2024-09-13T09:11:12.030770+0000","last_clean":"2024-09-13T09:11:12.030770+0000","last_became_active":"2024-09-13T09:07:13.864651+0000","last_became_peered":"2024-09-13T09:07:13.864651+0000","last_unstale":"2024-09-13T09:11:12.030770+0000","last_undegraded":"2024-09-13T09:11:12.030770+0000","last_fullsized":"2024-09-13T09:11:12.030770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'447","last_scrub_stamp":"2024-09-13T09:11:12.030454+0000","last_deep_scrub":"210'447","last_deep_scrub_stamp":"2024-09-13T09:11:12.030454+0000","last_clean_scrub_stamp":"2024-09-13T09:11:12.030454+0000","objects_scrubbed":14,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:41:50.134213+0000","scrub_duration":0.025469632999999998,"objects_trimmed":0,"snaptrim_duration":0.001256109,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2732,"num_read_kb":14647,"num_write":388,"num_write_kb":5830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":218,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:01.563398+0000","last_change":"2024-09-13T09:11:01.563398+0000","last_active":"2024-09-13T09:11:01.563398+0000","last_peered":"2024-09-13T09:11:01.563398+0000","last_clean":"2024-09-13T09:11:01.563398+0000","last_became_active":"2024-09-13T09:07:04.133316+0000","last_became_peered":"2024-09-13T09:07:04.133316+0000","last_unstale":"2024-09-13T09:11:01.563398+0000","last_undegraded":"2024-09-13T09:11:01.563398+0000","last_fullsized":"2024-09-13T09:11:01.563398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:01.563140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:01.563140+0000","last_clean_scrub_stamp":"2024-09-13T09:11:01.563140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:13:31.179427+0000","scrub_duration":0.0029921449999999999,"objects_trimmed":0,"snaptrim_duration":0.0026350789999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"210'630","reported_seq":6268,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.291622+0000","last_change":"2024-09-13T09:11:04.291622+0000","last_active":"2024-09-13T09:11:04.291622+0000","last_peered":"2024-09-13T09:11:04.291622+0000","last_clean":"2024-09-13T09:11:04.291622+0000","last_became_active":"2024-09-13T09:07:13.871396+0000","last_became_peered":"2024-09-13T09:07:13.871396+0000","last_unstale":"2024-09-13T09:11:04.291622+0000","last_undegraded":"2024-09-13T09:11:04.291622+0000","last_fullsized":"2024-09-13T09:11:04.291622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'630","last_scrub_stamp":"2024-09-13T09:11:04.291332+0000","last_deep_scrub":"210'630","last_deep_scrub_stamp":"2024-09-13T09:11:04.291332+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.291332+0000","objects_scrubbed":9,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:06:57.566099+0000","scrub_duration":0.017666608,"objects_trimmed":0,"snaptrim_duration":0.00043649599999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6810,"num_read_kb":26019,"num_write":814,"num_write_kb":6965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":219,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:01.891492+0000","last_change":"2024-09-13T09:11:01.891492+0000","last_active":"2024-09-13T09:11:01.891492+0000","last_peered":"2024-09-13T09:11:01.891492+0000","last_clean":"2024-09-13T09:11:01.891492+0000","last_became_active":"2024-09-13T09:07:04.189285+0000","last_became_peered":"2024-09-13T09:07:04.189285+0000","last_unstale":"2024-09-13T09:11:01.891492+0000","last_undegraded":"2024-09-13T09:11:01.891492+0000","last_fullsized":"2024-09-13T09:11:01.891492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:01.891225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:01.891225+0000","last_clean_scrub_stamp":"2024-09-13T09:11:01.891225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:41:05.422860+0000","scrub_duration":0.002855771,"objects_trimmed":0,"snaptrim_duration":0.00070164800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"210'725","reported_seq":3908,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:03.507995+0000","last_change":"2024-09-13T09:11:03.507995+0000","last_active":"2024-09-13T09:11:03.507995+0000","last_peered":"2024-09-13T09:11:03.507995+0000","last_clean":"2024-09-13T09:11:03.507995+0000","last_became_active":"2024-09-13T09:07:13.866654+0000","last_became_peered":"2024-09-13T09:07:13.866654+0000","last_unstale":"2024-09-13T09:11:03.507995+0000","last_undegraded":"2024-09-13T09:11:03.507995+0000","last_fullsized":"2024-09-13T09:11:03.507995+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'725","last_scrub_stamp":"2024-09-13T09:11:03.507714+0000","last_deep_scrub":"210'725","last_deep_scrub_stamp":"2024-09-13T09:11:03.507714+0000","last_clean_scrub_stamp":"2024-09-13T09:11:03.507714+0000","objects_scrubbed":8,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:15:38.375963+0000","scrub_duration":0.018454834,"objects_trimmed":0,"snaptrim_duration":0.0014721929999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4939,"num_read_kb":19757,"num_write":904,"num_write_kb":10484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"210'681","reported_seq":3565,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:58.282339+0000","last_change":"2024-09-13T09:10:58.282460+0000","last_active":"2024-09-13T09:10:58.282339+0000","last_peered":"2024-09-13T09:10:58.282339+0000","last_clean":"2024-09-13T09:10:58.282339+0000","last_became_active":"2024-09-13T09:10:36.917124+0000","last_became_peered":"2024-09-13T09:10:36.917124+0000","last_unstale":"2024-09-13T09:10:58.282339+0000","last_undegraded":"2024-09-13T09:10:58.282339+0000","last_fullsized":"2024-09-13T09:10:58.282339+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'681","last_scrub_stamp":"2024-09-13T09:10:58.282319+0000","last_deep_scrub":"210'681","last_deep_scrub_stamp":"2024-09-13T09:10:58.282319+0000","last_clean_scrub_stamp":"2024-09-13T09:10:58.282319+0000","objects_scrubbed":14,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:57.991183+0000","scrub_duration":0.021272119999999999,"objects_trimmed":0,"snaptrim_duration":0.0011757899999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":13,"num_read":3513,"num_read_kb":18416,"num_write":638,"num_write_kb":6216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":2344796,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"210'3","reported_seq":221,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:11.573384+0000","last_change":"2024-09-13T09:11:11.573384+0000","last_active":"2024-09-13T09:11:11.573384+0000","last_peered":"2024-09-13T09:11:11.573384+0000","last_clean":"2024-09-13T09:11:11.573384+0000","last_became_active":"2024-09-13T09:07:04.189750+0000","last_became_peered":"2024-09-13T09:07:04.189750+0000","last_unstale":"2024-09-13T09:11:11.573384+0000","last_undegraded":"2024-09-13T09:11:11.573384+0000","last_fullsized":"2024-09-13T09:11:11.573384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'3","last_scrub_stamp":"2024-09-13T09:11:11.573111+0000","last_deep_scrub":"210'3","last_deep_scrub_stamp":"2024-09-13T09:11:11.573111+0000","last_clean_scrub_stamp":"2024-09-13T09:11:11.573111+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:20.023637+0000","scrub_duration":0.0068124739999999998,"objects_trimmed":0,"snaptrim_duration":0.002061331,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"210'523","reported_seq":3824,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:11.869828+0000","last_change":"2024-09-13T09:11:11.869828+0000","last_active":"2024-09-13T09:11:11.869828+0000","last_peered":"2024-09-13T09:11:11.869828+0000","last_clean":"2024-09-13T09:11:11.869828+0000","last_became_active":"2024-09-13T09:10:19.192179+0000","last_became_peered":"2024-09-13T09:10:19.192179+0000","last_unstale":"2024-09-13T09:11:11.869828+0000","last_undegraded":"2024-09-13T09:11:11.869828+0000","last_fullsized":"2024-09-13T09:11:11.869828+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'523","last_scrub_stamp":"2024-09-13T09:11:11.869514+0000","last_deep_scrub":"210'523","last_deep_scrub_stamp":"2024-09-13T09:11:11.869514+0000","last_clean_scrub_stamp":"2024-09-13T09:11:11.869514+0000","objects_scrubbed":19,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:57:46.036506+0000","scrub_duration":0.034765302999999997,"objects_trimmed":0,"snaptrim_duration":0.00095688200000000005,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3585,"num_read_kb":19320,"num_write":505,"num_write_kb":9393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":6222896,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":59,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:58.760700+0000","last_change":"2024-09-13T09:10:58.760935+0000","last_active":"2024-09-13T09:10:58.760700+0000","last_peered":"2024-09-13T09:10:58.760700+0000","last_clean":"2024-09-13T09:10:58.760700+0000","last_became_active":"2024-09-13T09:10:19.191598+0000","last_became_peered":"2024-09-13T09:10:19.191598+0000","last_unstale":"2024-09-13T09:10:58.760700+0000","last_undegraded":"2024-09-13T09:10:58.760700+0000","last_fullsized":"2024-09-13T09:10:58.760700+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:10:58.760647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:10:58.760647+0000","last_clean_scrub_stamp":"2024-09-13T09:10:58.760647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:17.592464+0000","scrub_duration":0.004023591,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"210'432","reported_seq":2377,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:06.352417+0000","last_change":"2024-09-13T09:11:06.352417+0000","last_active":"2024-09-13T09:11:06.352417+0000","last_peered":"2024-09-13T09:11:06.352417+0000","last_clean":"2024-09-13T09:11:06.352417+0000","last_became_active":"2024-09-13T09:07:13.863065+0000","last_became_peered":"2024-09-13T09:07:13.863065+0000","last_unstale":"2024-09-13T09:11:06.352417+0000","last_undegraded":"2024-09-13T09:11:06.352417+0000","last_fullsized":"2024-09-13T09:11:06.352417+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'432","last_scrub_stamp":"2024-09-13T09:11:06.352132+0000","last_deep_scrub":"210'432","last_deep_scrub_stamp":"2024-09-13T09:11:06.352132+0000","last_clean_scrub_stamp":"2024-09-13T09:11:06.352132+0000","objects_scrubbed":10,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:24:05.270131+0000","scrub_duration":0.016779835,"objects_trimmed":0,"snaptrim_duration":0.00074152199999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2071,"num_read_kb":14898,"num_write":412,"num_write_kb":8289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":218,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:02.558028+0000","last_change":"2024-09-13T09:11:02.558028+0000","last_active":"2024-09-13T09:11:02.558028+0000","last_peered":"2024-09-13T09:11:02.558028+0000","last_clean":"2024-09-13T09:11:02.558028+0000","last_became_active":"2024-09-13T09:07:04.129030+0000","last_became_peered":"2024-09-13T09:07:04.129030+0000","last_unstale":"2024-09-13T09:11:02.558028+0000","last_undegraded":"2024-09-13T09:11:02.558028+0000","last_fullsized":"2024-09-13T09:11:02.558028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T09:11:02.557758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T09:11:02.557758+0000","last_clean_scrub_stamp":"2024-09-13T09:11:02.557758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:06.344243+0000","scrub_duration":0.0032222179999999998,"objects_trimmed":0,"snaptrim_duration":0.0022168130000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"210'415","reported_seq":2602,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:09.081149+0000","last_change":"2024-09-13T09:11:09.081149+0000","last_active":"2024-09-13T09:11:09.081149+0000","last_peered":"2024-09-13T09:11:09.081149+0000","last_clean":"2024-09-13T09:11:09.081149+0000","last_became_active":"2024-09-13T09:07:13.860671+0000","last_became_peered":"2024-09-13T09:07:13.860671+0000","last_unstale":"2024-09-13T09:11:09.081149+0000","last_undegraded":"2024-09-13T09:11:09.081149+0000","last_fullsized":"2024-09-13T09:11:09.081149+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'415","last_scrub_stamp":"2024-09-13T09:11:09.080850+0000","last_deep_scrub":"210'415","last_deep_scrub_stamp":"2024-09-13T09:11:09.080850+0000","last_clean_scrub_stamp":"2024-09-13T09:11:09.080850+0000","objects_scrubbed":10,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:25:16.304447+0000","scrub_duration":0.019439997000000001,"objects_trimmed":0,"snaptrim_duration":0.001037313,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2540,"num_read_kb":12233,"num_write":423,"num_write_kb":3733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"210'2","reported_seq":220,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.597534+0000","last_change":"2024-09-13T09:11:04.597534+0000","last_active":"2024-09-13T09:11:04.597534+0000","last_peered":"2024-09-13T09:11:04.597534+0000","last_clean":"2024-09-13T09:11:04.597534+0000","last_became_active":"2024-09-13T09:07:04.350852+0000","last_became_peered":"2024-09-13T09:07:04.350852+0000","last_unstale":"2024-09-13T09:11:04.597534+0000","last_undegraded":"2024-09-13T09:11:04.597534+0000","last_fullsized":"2024-09-13T09:11:04.597534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'2","last_scrub_stamp":"2024-09-13T09:11:04.597248+0000","last_deep_scrub":"210'2","last_deep_scrub_stamp":"2024-09-13T09:11:04.597248+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.597248+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:39:56.082283+0000","scrub_duration":0.0055551189999999999,"objects_trimmed":0,"snaptrim_duration":0.00065551900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"210'119","reported_seq":349,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.913895+0000","last_change":"2024-09-13T09:11:04.914128+0000","last_active":"2024-09-13T09:11:04.913895+0000","last_peered":"2024-09-13T09:11:04.913895+0000","last_clean":"2024-09-13T09:11:04.913895+0000","last_became_active":"2024-09-13T09:07:53.861729+0000","last_became_peered":"2024-09-13T09:07:53.861729+0000","last_unstale":"2024-09-13T09:11:04.913895+0000","last_undegraded":"2024-09-13T09:11:04.913895+0000","last_fullsized":"2024-09-13T09:11:04.913895+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'119","last_scrub_stamp":"2024-09-13T09:11:04.913842+0000","last_deep_scrub":"210'119","last_deep_scrub_stamp":"2024-09-13T09:11:04.913842+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.913842+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:57.881164+0000","scrub_duration":0.0085814419999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"210'893","reported_seq":6177,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.128803+0000","last_change":"2024-09-13T09:11:04.128803+0000","last_active":"2024-09-13T09:11:04.128803+0000","last_peered":"2024-09-13T09:11:04.128803+0000","last_clean":"2024-09-13T09:11:04.128803+0000","last_became_active":"2024-09-13T09:10:19.200589+0000","last_became_peered":"2024-09-13T09:10:19.200589+0000","last_unstale":"2024-09-13T09:11:04.128803+0000","last_undegraded":"2024-09-13T09:11:04.128803+0000","last_fullsized":"2024-09-13T09:11:04.128803+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'893","last_scrub_stamp":"2024-09-13T09:11:04.128368+0000","last_deep_scrub":"210'893","last_deep_scrub_stamp":"2024-09-13T09:11:04.128368+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.128368+0000","objects_scrubbed":12,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:39:53.087941+0000","scrub_duration":0.02779214,"objects_trimmed":0,"snaptrim_duration":0.0018329830000000001,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6854,"num_read_kb":23202,"num_write":1277,"num_write_kb":11885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":9956154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"210'602","reported_seq":3666,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:16.604833+0000","last_change":"2024-09-13T09:11:16.604833+0000","last_active":"2024-09-13T09:11:16.604833+0000","last_peered":"2024-09-13T09:11:16.604833+0000","last_clean":"2024-09-13T09:11:16.604833+0000","last_became_active":"2024-09-13T09:07:13.861699+0000","last_became_peered":"2024-09-13T09:07:13.861699+0000","last_unstale":"2024-09-13T09:11:16.604833+0000","last_undegraded":"2024-09-13T09:11:16.604833+0000","last_fullsized":"2024-09-13T09:11:16.604833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'602","last_scrub_stamp":"2024-09-13T09:11:16.604534+0000","last_deep_scrub":"210'602","last_deep_scrub_stamp":"2024-09-13T09:11:16.604534+0000","last_clean_scrub_stamp":"2024-09-13T09:11:16.604534+0000","objects_scrubbed":8,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:22:12.744010+0000","scrub_duration":0.017323160000000001,"objects_trimmed":0,"snaptrim_duration":0.00099698400000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4866,"num_read_kb":16848,"num_write":688,"num_write_kb":9254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"210'594","reported_seq":3126,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:27.992018+0000","last_change":"2024-09-13T09:11:27.992249+0000","last_active":"2024-09-13T09:11:27.992018+0000","last_peered":"2024-09-13T09:11:27.992018+0000","last_clean":"2024-09-13T09:11:27.992018+0000","last_became_active":"2024-09-13T09:08:21.087760+0000","last_became_peered":"2024-09-13T09:08:21.087760+0000","last_unstale":"2024-09-13T09:11:27.992018+0000","last_undegraded":"2024-09-13T09:11:27.992018+0000","last_fullsized":"2024-09-13T09:11:27.992018+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'594","last_scrub_stamp":"2024-09-13T09:11:27.991929+0000","last_deep_scrub":"210'594","last_deep_scrub_stamp":"2024-09-13T09:11:27.991929+0000","last_clean_scrub_stamp":"2024-09-13T09:11:27.991929+0000","objects_scrubbed":4,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:37.438322+0000","scrub_duration":0.01048579,"objects_trimmed":0,"snaptrim_duration":0.00074287699999999997,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4001,"num_read_kb":22630,"num_write":547,"num_write_kb":17993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":81,"num_bytes_recovered":12860287,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"210'438","reported_seq":3900,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:04.475858+0000","last_change":"2024-09-13T09:11:04.476097+0000","last_active":"2024-09-13T09:11:04.475858+0000","last_peered":"2024-09-13T09:11:04.475858+0000","last_clean":"2024-09-13T09:11:04.475858+0000","last_became_active":"2024-09-13T09:09:42.306043+0000","last_became_peered":"2024-09-13T09:09:42.306043+0000","last_unstale":"2024-09-13T09:11:04.475858+0000","last_undegraded":"2024-09-13T09:11:04.475858+0000","last_fullsized":"2024-09-13T09:11:04.475858+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'438","last_scrub_stamp":"2024-09-13T09:11:04.475792+0000","last_deep_scrub":"210'438","last_deep_scrub_stamp":"2024-09-13T09:11:04.475792+0000","last_clean_scrub_stamp":"2024-09-13T09:11:04.475792+0000","objects_scrubbed":10,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:37:09.660309+0000","scrub_duration":0.018211029,"objects_trimmed":0,"snaptrim_duration":0.0013662279999999999,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3594,"num_read_kb":33920,"num_write":421,"num_write_kb":17807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":16921892,"num_keys_recovered":13,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"210'372","reported_seq":17082,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:56.293946+0000","last_change":"2024-09-13T09:10:56.294059+0000","last_active":"2024-09-13T09:10:56.293946+0000","last_peered":"2024-09-13T09:10:56.293946+0000","last_clean":"2024-09-13T09:10:56.293946+0000","last_became_active":"2024-09-13T09:10:35.919376+0000","last_became_peered":"2024-09-13T09:10:35.919376+0000","last_unstale":"2024-09-13T09:10:56.293946+0000","last_undegraded":"2024-09-13T09:10:56.293946+0000","last_fullsized":"2024-09-13T09:10:56.293946+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'372","last_scrub_stamp":"2024-09-13T09:10:56.293925+0000","last_deep_scrub":"210'372","last_deep_scrub_stamp":"2024-09-13T09:10:56.293925+0000","last_clean_scrub_stamp":"2024-09-13T09:10:56.293925+0000","objects_scrubbed":12,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:05:32.732009+0000","scrub_duration":0.014972901,"objects_trimmed":0,"snaptrim_duration":0.00088103200000000002,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":16596,"num_read_kb":27933,"num_write":361,"num_write_kb":15673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":374,"num_bytes_recovered":39814577,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"210'420","reported_seq":3159,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:19.624764+0000","last_change":"2024-09-13T09:11:19.625031+0000","last_active":"2024-09-13T09:11:19.624764+0000","last_peered":"2024-09-13T09:11:19.624764+0000","last_clean":"2024-09-13T09:11:19.624764+0000","last_became_active":"2024-09-13T09:07:50.845769+0000","last_became_peered":"2024-09-13T09:07:50.845769+0000","last_unstale":"2024-09-13T09:11:19.624764+0000","last_undegraded":"2024-09-13T09:11:19.624764+0000","last_fullsized":"2024-09-13T09:11:19.624764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'420","last_scrub_stamp":"2024-09-13T09:11:19.624708+0000","last_deep_scrub":"210'420","last_deep_scrub_stamp":"2024-09-13T09:11:19.624708+0000","last_clean_scrub_stamp":"2024-09-13T09:11:19.624708+0000","objects_scrubbed":10,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:32:41.671376+0000","scrub_duration":0.019295928,"objects_trimmed":0,"snaptrim_duration":0.001690069,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":3095,"num_read_kb":9820,"num_write":360,"num_write_kb":5232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":454477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"210'893","reported_seq":7393,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:57.491271+0000","last_change":"2024-09-13T09:10:57.491494+0000","last_active":"2024-09-13T09:10:57.491271+0000","last_peered":"2024-09-13T09:10:57.491271+0000","last_clean":"2024-09-13T09:10:57.491271+0000","last_became_active":"2024-09-13T09:10:35.914456+0000","last_became_peered":"2024-09-13T09:10:35.914456+0000","last_unstale":"2024-09-13T09:10:57.491271+0000","last_undegraded":"2024-09-13T09:10:57.491271+0000","last_fullsized":"2024-09-13T09:10:57.491271+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'893","last_scrub_stamp":"2024-09-13T09:10:57.491220+0000","last_deep_scrub":"210'893","last_deep_scrub_stamp":"2024-09-13T09:10:57.491220+0000","last_clean_scrub_stamp":"2024-09-13T09:10:57.491220+0000","objects_scrubbed":9,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:57:11.835289+0000","scrub_duration":0.018083443000000001,"objects_trimmed":0,"snaptrim_duration":0.001663692,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":9292,"num_read_kb":23776,"num_write":1223,"num_write_kb":6169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":1618626,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"210'490","reported_seq":5203,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:02.258332+0000","last_change":"2024-09-13T09:11:02.258568+0000","last_active":"2024-09-13T09:11:02.258332+0000","last_peered":"2024-09-13T09:11:02.258332+0000","last_clean":"2024-09-13T09:11:02.258332+0000","last_became_active":"2024-09-13T09:10:35.915413+0000","last_became_peered":"2024-09-13T09:10:35.915413+0000","last_unstale":"2024-09-13T09:11:02.258332+0000","last_undegraded":"2024-09-13T09:11:02.258332+0000","last_fullsized":"2024-09-13T09:11:02.258332+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'490","last_scrub_stamp":"2024-09-13T09:11:02.258279+0000","last_deep_scrub":"210'490","last_deep_scrub_stamp":"2024-09-13T09:11:02.258279+0000","last_clean_scrub_stamp":"2024-09-13T09:11:02.258279+0000","objects_scrubbed":13,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:58:47.729702+0000","scrub_duration":0.025685860000000001,"objects_trimmed":0,"snaptrim_duration":0.001550994,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5016,"num_read_kb":17262,"num_write":425,"num_write_kb":6360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":565,"num_bytes_recovered":11317810,"num_keys_recovered":88,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"210'533","reported_seq":2934,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:02.921486+0000","last_change":"2024-09-13T09:11:02.921712+0000","last_active":"2024-09-13T09:11:02.921486+0000","last_peered":"2024-09-13T09:11:02.921486+0000","last_clean":"2024-09-13T09:11:02.921486+0000","last_became_active":"2024-09-13T09:08:09.863231+0000","last_became_peered":"2024-09-13T09:08:09.863231+0000","last_unstale":"2024-09-13T09:11:02.921486+0000","last_undegraded":"2024-09-13T09:11:02.921486+0000","last_fullsized":"2024-09-13T09:11:02.921486+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'533","last_scrub_stamp":"2024-09-13T09:11:02.921433+0000","last_deep_scrub":"210'533","last_deep_scrub_stamp":"2024-09-13T09:11:02.921433+0000","last_clean_scrub_stamp":"2024-09-13T09:11:02.921433+0000","objects_scrubbed":15,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:53:22.794757+0000","scrub_duration":0.025615438000000001,"objects_trimmed":0,"snaptrim_duration":0.0011980339999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":2903,"num_read_kb":17169,"num_write":491,"num_write_kb":9789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":9077216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"210'1196","reported_seq":4432,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:23.636636+0000","last_change":"2024-09-13T09:11:23.636869+0000","last_active":"2024-09-13T09:11:23.636636+0000","last_peered":"2024-09-13T09:11:23.636636+0000","last_clean":"2024-09-13T09:11:23.636636+0000","last_became_active":"2024-09-13T09:07:13.858663+0000","last_became_peered":"2024-09-13T09:07:13.858663+0000","last_unstale":"2024-09-13T09:11:23.636636+0000","last_undegraded":"2024-09-13T09:11:23.636636+0000","last_fullsized":"2024-09-13T09:11:23.636636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'1196","last_scrub_stamp":"2024-09-13T09:11:23.636576+0000","last_deep_scrub":"210'1196","last_deep_scrub_stamp":"2024-09-13T09:11:23.636576+0000","last_clean_scrub_stamp":"2024-09-13T09:11:23.636576+0000","objects_scrubbed":9,"log_size":1196,"log_dups_size":0,"ondisk_log_size":1196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:05:49.091857+0000","scrub_duration":0.021054897,"objects_trimmed":0,"snaptrim_duration":0.001456372,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":7816,"num_read_kb":12367,"num_write":1664,"num_write_kb":5387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"210'725","reported_seq":4634,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:10:58.292323+0000","last_change":"2024-09-13T09:10:58.292581+0000","last_active":"2024-09-13T09:10:58.292323+0000","last_peered":"2024-09-13T09:10:58.292323+0000","last_clean":"2024-09-13T09:10:58.292323+0000","last_became_active":"2024-09-13T09:07:13.863071+0000","last_became_peered":"2024-09-13T09:07:13.863071+0000","last_unstale":"2024-09-13T09:10:58.292323+0000","last_undegraded":"2024-09-13T09:10:58.292323+0000","last_fullsized":"2024-09-13T09:10:58.292323+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'725","last_scrub_stamp":"2024-09-13T09:10:58.292265+0000","last_deep_scrub":"210'725","last_deep_scrub_stamp":"2024-09-13T09:10:58.292265+0000","last_clean_scrub_stamp":"2024-09-13T09:10:58.292265+0000","objects_scrubbed":7,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:05:09.243645+0000","scrub_duration":0.01780489,"objects_trimmed":0,"snaptrim_duration":0.0012075230000000001,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5826,"num_read_kb":14063,"num_write":971,"num_write_kb":11379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"210'603","reported_seq":4180,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-13T09:11:22.679054+0000","last_change":"2024-09-13T09:11:22.679291+0000","last_active":"2024-09-13T09:11:22.679054+0000","last_peered":"2024-09-13T09:11:22.679054+0000","last_clean":"2024-09-13T09:11:22.679054+0000","last_became_active":"2024-09-13T09:10:31.325692+0000","last_became_peered":"2024-09-13T09:10:31.325692+0000","last_unstale":"2024-09-13T09:11:22.679054+0000","last_undegraded":"2024-09-13T09:11:22.679054+0000","last_fullsized":"2024-09-13T09:11:22.679054+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'603","last_scrub_stamp":"2024-09-13T09:11:22.678998+0000","last_deep_scrub":"210'603","last_deep_scrub_stamp":"2024-09-13T09:11:22.678998+0000","last_clean_scrub_stamp":"2024-09-13T09:11:22.678998+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:19:25.196462+0000","scrub_duration":0.028838308,"objects_trimmed":0,"snaptrim_duration":0.0015796709999999999,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":3375,"num_read_kb":13502,"num_write":606,"num_write_kb":7572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":227,"num_bytes_recovered":3457983,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":340,"num_object_clones":0,"num_object_copies":680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":340,"num_whiteouts":336,"num_read":210850,"num_read_kb":690514,"num_write":25042,"num_write_kb":338108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3692,"num_bytes_recovered":209085124,"num_keys_recovered":477,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0},"log_size":21760,"ondisk_log_size":21760,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":181,"seq":777389080594,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":2472,"kb_used_omap":19,"kb_used_meta":28844,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":20246,"internal_metadata":29536490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:22 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.78800000000000003}]},{"osd":1,"last update":"Fri Sep 13 09:11:23 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.88800000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.726,"5min":2.726,"15min":2.726},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.006}]},{"osd":3,"last update":"Fri Sep 13 09:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":1.149},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.1930000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:11:22 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.8690000000000002,"5min":2.8690000000000002,"15min":2.8690000000000002},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":3.0550000000000002},"last":1.038}]},{"osd":6,"last update":"Fri Sep 13 09:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":1.1279999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509500,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69684,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302156,"statfs":{"total":96636764160,"available":96565407744,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.23100000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.91600000000000004,"5min":1.361,"15min":1.361},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63600000000000001,"15min":0.64700000000000002},"min":{"1min":0.34000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.97099999999999997,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":5.7080000000000002,"5min":5.7080000000000002,"15min":5.7080000000000002},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63200000000000001,"15min":0.64800000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.996,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.63500000000000001},"min":{"1min":0.35699999999999998,"5min":0.253,"15min":0.253},"max":{"1min":1.004,"5min":1.444,"15min":1.444},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.68700000000000006,"15min":0.60299999999999998},"min":{"1min":0.32100000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":5.8109999999999999,"5min":5.8109999999999999,"15min":5.8109999999999999},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.84099999999999997,"15min":0.84899999999999998},"min":{"1min":0.52000000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":0.91400000000000003,"15min":0.878},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":5.8449999999999998,"5min":5.8449999999999998,"15min":5.8449999999999998},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.84499999999999997,"15min":0.83099999999999996},"min":{"1min":0.47699999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.80700000000000005}]},{"osd":6,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":0.96899999999999997,"15min":0.92200000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":5.6260000000000003,"5min":5.6260000000000003,"15min":5.6260000000000003},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.92000000000000004,"15min":0.91900000000000004},"min":{"1min":0.71299999999999997,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.375,"5min":1.379,"15min":1.379},"last":0.98199999999999998}]},{"osd":7,"last update":"Fri Sep 13 09:11:19 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.016}]}]},{"osd":2,"up_from":15,"seq":64424509499,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69680,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302160,"statfs":{"total":96636764160,"available":96565411840,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.63700000000000001,"15min":0.68799999999999994},"min":{"1min":0.246,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.91300000000000003,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.70299999999999996,"15min":0.73899999999999999},"min":{"1min":0.38,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84699999999999998,"5min":1.353,"15min":1.353},"last":0.96999999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.61699999999999999,"15min":0.64900000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.80800000000000005,"5min":1.484,"15min":1.484},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.73799999999999999,"15min":0.77600000000000002},"min":{"1min":0.33600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":1.0109999999999999,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.73199999999999998,"15min":0.79300000000000004},"min":{"1min":0.32500000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.94999999999999996,"5min":1.681,"15min":1.681},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.80400000000000005,"15min":0.88800000000000001},"min":{"1min":0.41599999999999998,"5min":0.245,"15min":0.245},"max":{"1min":1.179,"5min":1.401,"15min":1.401},"last":1.1040000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.91600000000000004,"15min":1.008},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1759999999999999,"5min":1.998,"15min":1.998},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.89700000000000002,"15min":0.94199999999999995},"min":{"1min":0.44900000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1830000000000001,"5min":1.651,"15min":1.651},"last":0.85299999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.93200000000000005,"15min":0.97199999999999998},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.127,"5min":1.506,"15min":1.506},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.95799999999999996,"15min":1.03},"min":{"1min":0.438,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.254,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:11:02 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":1.006,"15min":1.0649999999999999},"min":{"1min":0.53900000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.494,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":1.0429999999999999,"15min":1.109},"min":{"1min":0.59499999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2829999999999999,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.2390000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.282},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.3109999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56312,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94315528,"statfs":{"total":96636764160,"available":96579100672,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.26700000000000002,"5min":0.221,"15min":0.221},"max":{"1min":1.3049999999999999,"5min":2.2650000000000001,"15min":2.2650000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.63900000000000001},"min":{"1min":0.33000000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.86799999999999999}]},{"osd":2,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.63900000000000001,"15min":0.64900000000000002},"min":{"1min":0.26900000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.71299999999999997,"15min":0.71499999999999997},"min":{"1min":0.32400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":2.7010000000000001},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.69099999999999995,"15min":0.72399999999999998},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.73499999999999999,"15min":0.76100000000000001},"min":{"1min":0.40200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.72799999999999998}]},{"osd":4,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.86899999999999999,"15min":0.85599999999999998},"min":{"1min":0.55600000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.7930000000000001,"5min":3.7930000000000001,"15min":3.7930000000000001},"last":1.03},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.90800000000000003,"15min":0.93700000000000006},"min":{"1min":0.66700000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":0.95799999999999996,"15min":0.98399999999999999},"min":{"1min":0.51800000000000002,"5min":0.498,"15min":0.498},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":1.083},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.96099999999999997,"15min":0.96699999999999997},"min":{"1min":0.59799999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.351,"5min":3.351,"15min":3.351},"last":0.88800000000000001}]},{"osd":6,"last update":"Fri Sep 13 09:11:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.93500000000000005,"15min":0.95199999999999996},"min":{"1min":0.54000000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.75,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":1.014},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.014,"15min":1.0509999999999999},"min":{"1min":0.61799999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8899999999999999,"5min":2.2290000000000001,"15min":2.2290000000000001},"last":1.107}]},{"osd":7,"last update":"Fri Sep 13 09:11:23 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.228,"15min":1.228},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.159},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.99299999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37680,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334160,"statfs":{"total":96636764160,"available":96598179840,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56799999999999995,"15min":0.59499999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.61399999999999999,"15min":0.63},"min":{"1min":0.378,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.78300000000000003}]},{"osd":2,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.64200000000000002,"15min":0.69399999999999995},"min":{"1min":0.33800000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96199999999999997,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.623},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.751,"15min":0.85299999999999998},"min":{"1min":0.28199999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.3140000000000001,"5min":3.855,"15min":3.855},"last":1.052}]},{"osd":3,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.80400000000000005},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":1.107,"5min":2.7999999999999998,"15min":2.7999999999999998},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78900000000000003,"15min":0.91300000000000003},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.4390000000000001,"5min":3.6509999999999998,"15min":3.6509999999999998},"last":0.95099999999999996}]},{"osd":4,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.50900000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.3660000000000001,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.872,"15min":0.93100000000000005},"min":{"1min":0.61199999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.139,"5min":2.4860000000000002,"15min":2.4860000000000002},"last":1.1859999999999999}]},{"osd":5,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.93999999999999995,"15min":0.97399999999999998},"min":{"1min":0.70599999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.583,"5min":2.532,"15min":2.532},"last":1.266},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.056,"15min":1.121},"min":{"1min":0.70999999999999996,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.698,"5min":2.181,"15min":2.181},"last":1.228}]},{"osd":6,"last update":"Fri Sep 13 09:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.0169999999999999,"15min":1.111},"min":{"1min":0.61099999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.228,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.012},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.98099999999999998,"15min":1.0609999999999999},"min":{"1min":0.54700000000000004,"5min":0.44,"15min":0.44},"max":{"1min":1.347,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.3420000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:11:18 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.157},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.1140000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542204,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37684,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334156,"statfs":{"total":96636764160,"available":96598175744,"internally_reserved":0,"allocated":2535424,"data_stored":1226484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.79800000000000004,"15min":0.82199999999999995},"min":{"1min":0.54400000000000004,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":1.3089999999999999,"5min":1.552,"15min":1.552},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.80900000000000005,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.1599999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.2829999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.83299999999999996,"15min":0.80100000000000005},"min":{"1min":0.50600000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2929999999999999,"5min":5.4219999999999997,"15min":5.4219999999999997},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.89400000000000002,"15min":0.91600000000000004},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4159999999999999,"5min":2.706,"15min":2.706},"last":1.3240000000000001}]},{"osd":2,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.93899999999999995,"15min":0.89800000000000002},"min":{"1min":0.497,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.325,"5min":8.3499999999999996,"15min":8.3499999999999996},"last":1.3720000000000001},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.96299999999999997,"15min":0.93799999999999994},"min":{"1min":0.61099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.22,"5min":8.2989999999999995,"15min":8.2989999999999995},"last":1.0840000000000001}]},{"osd":3,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.92500000000000004,"15min":0.94499999999999995},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.145,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.92800000000000005,"15min":0.92400000000000004},"min":{"1min":0.45100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.2230000000000001}]},{"osd":4,"last update":"Fri Sep 13 09:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.96499999999999997,"15min":0.98899999999999999},"min":{"1min":0.53700000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2949999999999999,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.91000000000000003,"15min":0.95999999999999996},"min":{"1min":0.39200000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.379,"5min":1.7,"15min":1.7},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Sep 13 09:11:01 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.054,"15min":1.083},"min":{"1min":0.52800000000000002,"5min":0.497,"15min":0.497},"max":{"1min":1.6339999999999999,"5min":1.776,"15min":1.776},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0429999999999999,"15min":1.0629999999999999},"min":{"1min":0.41299999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.581,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.476}]},{"osd":7,"last update":"Fri Sep 13 09:11:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.97699999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542203,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47536,"kb_used_data":2472,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324304,"statfs":{"total":96636764160,"available":96588087296,"internally_reserved":0,"allocated":2531328,"data_stored":1226465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.2749999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80200000000000005,"15min":0.83499999999999996},"min":{"1min":0.31,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.1379999999999999,"5min":2.649,"15min":2.649},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Sep 13 09:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.76500000000000001,"15min":0.74299999999999999},"min":{"1min":0.35499999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.83799999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.135,"5min":1.3,"15min":1.3},"last":1.177}]},{"osd":2,"last update":"Fri Sep 13 09:11:11 2024","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90000000000000002,"15min":0.83799999999999997},"min":{"1min":0.45900000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.2769999999999999,"5min":3.5819999999999999,"15min":3.5819999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.93100000000000005,"15min":0.92300000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.206,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.1459999999999999}]},{"osd":3,"last update":"Fri Sep 13 09:11:11 2024","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.94999999999999996,"15min":0.879},"min":{"1min":0.48799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.409,"5min":3.6749999999999998,"15min":3.6749999999999998},"last":1.2350000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.98199999999999998,"15min":0.97899999999999998},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.452,"5min":1.573,"15min":1.573},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.91900000000000004,"15min":0.86399999999999999},"min":{"1min":0.377,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.6830000000000001,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.006},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0009999999999999,"15min":0.90400000000000003},"min":{"1min":0.41299999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.756,"5min":3.7639999999999998,"15min":3.7639999999999998},"last":0.89500000000000002}]},{"osd":6,"last update":"Fri Sep 13 09:11:05 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.0069999999999999,"15min":0.95399999999999996},"min":{"1min":0.40200000000000002,"5min":0.375,"15min":0.375},"max":{"1min":1.522,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0449999999999999,"15min":1.04},"min":{"1min":0.442,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.792,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.125}]},{"osd":7,"last update":"Fri Sep 13 09:11:20 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.085}]}]},{"osd":5,"up_from":13,"seq":55834574909,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48120,"kb_used_data":3056,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323720,"statfs":{"total":96636764160,"available":96587489280,"internally_reserved":0,"allocated":3129344,"data_stored":1816852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.79600000000000004,"15min":0.79700000000000004},"min":{"1min":0.48199999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.1439999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.97799999999999998,"15min":1.1319999999999999},"min":{"1min":0.622,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.194,"5min":7.9740000000000002,"15min":7.9740000000000002},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.78100000000000003,"15min":0.82899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.28,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.874,"15min":0.88600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2809999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.95399999999999996,"15min":1.1080000000000001},"min":{"1min":0.49199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.155,"5min":8.1400000000000006,"15min":8.1400000000000006},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.86399999999999999,"15min":0.88600000000000001},"min":{"1min":0.68600000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0940000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.97699999999999998}]},{"osd":3,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.01,"15min":1.173},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.1679999999999999,"5min":7.9050000000000002,"15min":7.9050000000000002},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.93400000000000005,"15min":0.96599999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2609999999999999,"5min":1.585,"15min":1.585},"last":1.0309999999999999}]},{"osd":4,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.97799999999999998,"15min":1.018},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.321,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.93100000000000005,"5min":1.147,"15min":1.345},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.417,"5min":8.0850000000000009,"15min":8.0850000000000009},"last":1.075}]},{"osd":6,"last update":"Fri Sep 13 09:11:07 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.98399999999999999,"15min":0.98599999999999999},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.391,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.0529999999999999,"15min":1.0820000000000001},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.518,"5min":1.899,"15min":1.899},"last":1.1080000000000001}]},{"osd":7,"last update":"Fri Sep 13 09:11:20 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.1279999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10333,"internal_metadata":0}]}} 2024-09-13T09:11:39.482 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T09:11:39.800 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T09:11:39.800 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T09:11:39.800 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T09:11:39.800 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:11:39.882 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T09:11:39.882 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T09:11:39.882 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:11:41.494 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-13T09:11:45.984 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T09:11:45.984 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T09:11:45.984 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:11:46.045 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T09:11:46.046 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T09:11:46.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:11:47.401 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T09:11:47.402 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-13T09:11:52.147 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T09:11:52.147 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T09:11:52.147 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:11:52.244 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T09:11:52.244 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T09:11:52.244 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:11:53.108 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-13T09:11:53.108 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T09:11:58.346 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T09:11:58.346 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T09:11:58.346 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:11:58.418 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T09:11:58.418 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T09:11:58.418 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:11:58.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~6s 2024-09-13T09:12:04.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-13T09:12:04.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-13T09:12:04.521 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T09:12:04.521 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T09:12:04.522 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T09:12:04.522 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:12:04.550 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T09:12:04.550 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T09:12:04.551 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:12:04.582 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T09:12:04.583 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T09:12:04.583 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T09:12:04.583 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:12:04.611 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T09:12:04.611 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T09:12:04.611 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:12:04.671 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T09:12:04.672 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T09:12:04.672 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T09:12:04.689 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T09:12:04.689 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T09:12:04.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-13T09:12:04.731 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:12:04.731 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T09:12:05.003 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:12:05.003 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T09:12:05.213 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:12:05.213 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T09:12:05.422 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:12:05.422 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T09:12:05.614 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:12:05.614 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T09:12:05.918 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:12:05.918 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T09:12:06.064 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:12:06.064 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T09:12:06.223 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:12:06.223 DEBUG:teuthology.orchestra.run.smithi060:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T09:12:06.373 INFO:tasks.ceph:Archiving mon data... 2024-09-13T09:12:06.374 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/data/mon.a.tgz 2024-09-13T09:12:06.374 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-13T09:12:06.381 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.wJZWOoV6F8 2024-09-13T09:12:06.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wJZWOoV6F8 2024-09-13T09:12:06.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.wJZWOoV6F8 2024-09-13T09:12:06.608 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.wJZWOoV6F8 is 471KB 2024-09-13T09:12:06.669 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.wJZWOoV6F8 2024-09-13T09:12:06.674 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/data/mon.c.tgz 2024-09-13T09:12:06.675 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-13T09:12:06.723 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.nL1icCqrIV 2024-09-13T09:12:06.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.nL1icCqrIV 2024-09-13T09:12:06.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.nL1icCqrIV 2024-09-13T09:12:06.988 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.nL1icCqrIV is 633KB 2024-09-13T09:12:07.049 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.nL1icCqrIV 2024-09-13T09:12:07.055 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/data/mon.b.tgz 2024-09-13T09:12:07.055 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-09-13T09:12:07.061 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.GXRvMsZvEW 2024-09-13T09:12:07.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.GXRvMsZvEW 2024-09-13T09:12:07.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.GXRvMsZvEW 2024-09-13T09:12:07.330 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.GXRvMsZvEW is 633KB 2024-09-13T09:12:07.395 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.GXRvMsZvEW 2024-09-13T09:12:07.400 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T09:12:07.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:12:07.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T09:12:07.458 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T09:12:07.458 DEBUG:teuthology.orchestra.run.smithi031:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:12:07.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T09:12:07.482 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:12:07.529 INFO:teuthology.orchestra.run.smithi060.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T09:12:07.533 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T09:12:07.535 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/remote/smithi031/crash 2024-09-13T09:12:07.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:12:07.548 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/remote/smithi060/crash 2024-09-13T09:12:07.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T09:12:07.585 INFO:tasks.ceph:Compressing logs... 2024-09-13T09:12:07.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:12:07.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:12:09.321 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-13T09:12:20.936 INFO:tasks.ceph:Archiving logs... 2024-09-13T09:12:20.936 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/remote/smithi031/log 2024-09-13T09:12:20.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:12:22.274 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/remote/smithi060/log 2024-09-13T09:12:22.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T09:12:23.193 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T09:12:23.203 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T09:12:23.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:12:23.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T09:12:23.289 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T09:12:23.299 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T09:12:23.299 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:12:23.302 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T09:12:23.312 INFO:teuthology.orchestra.run.smithi031.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:12:23.312 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================== 2024-09-13T09:12:23.312 INFO:teuthology.orchestra.run.smithi031.stdout:+hv01.front.sepi 67.205.162.81 3 u 11 64 377 0.074 -0.524 0.590 2024-09-13T09:12:23.312 INFO:teuthology.orchestra.run.smithi031.stdout:+hv02.front.sepi 96.245.170.99 3 u 15 64 377 0.077 +0.071 0.598 2024-09-13T09:12:23.312 INFO:teuthology.orchestra.run.smithi031.stdout:*hv03.front.sepi 74.6.168.72 3 u 12 64 377 0.078 -1.267 0.800 2024-09-13T09:12:23.313 INFO:teuthology.orchestra.run.smithi031.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:12:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T09:12:23.314 INFO:teuthology.orchestra.run.smithi060.stdout:============================================================================== 2024-09-13T09:12:23.314 INFO:teuthology.orchestra.run.smithi060.stdout:*hv01.front.sepi 67.205.162.81 3 u 14 64 377 0.069 -0.571 0.283 2024-09-13T09:12:23.314 INFO:teuthology.orchestra.run.smithi060.stdout:+hv02.front.sepi 96.245.170.99 3 u 11 64 377 0.074 +0.378 0.234 2024-09-13T09:12:23.314 INFO:teuthology.orchestra.run.smithi060.stdout:+hv03.front.sepi 74.6.168.72 3 u 15 64 377 0.073 -0.018 0.134 2024-09-13T09:12:23.314 INFO:teuthology.orchestra.run.smithi060.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T09:12:23.315 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T09:12:23.325 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T09:12:23.325 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T09:12:23.336 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T09:12:23.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T09:12:23.358 INFO:teuthology.task.internal:Duration was 942.947275 seconds 2024-09-13T09:12:23.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T09:12:23.370 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T09:12:23.370 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:12:23.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T09:12:23.411 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T09:12:23.411 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-13T09:12:23.412 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:12:23.461 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-09-13T09:12:23.461 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T09:12:23.471 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T09:12:23.471 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:12:23.504 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T09:12:23.542 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T09:12:23.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:12:23.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T09:12:23.656 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T09:12:23.667 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T09:12:23.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:12:23.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T09:12:23.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T09:12:23.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:12:23.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:12:23.731 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-13T09:12:23.744 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-09-13T09:12:23.764 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:12:23.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:12:23.799 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T09:12:23.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:12:23.813 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T09:12:23.823 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T09:12:23.824 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/remote/smithi031 2024-09-13T09:12:23.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:12:23.870 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7901010/remote/smithi060 2024-09-13T09:12:23.870 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T09:12:23.894 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T09:12:23.895 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:12:23.908 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T09:12:23.937 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T09:12:23.949 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T09:12:23.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T09:12:23.970 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T09:12:23.970 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:12:23.972 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T09:12:23.977 INFO:teuthology.orchestra.run.smithi031.stdout: 658462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:12 /home/ubuntu/cephtest 2024-09-13T09:12:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: 658471 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 09:12 /home/ubuntu/cephtest 2024-09-13T09:12:23.987 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T09:12:23.997 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T09:12:24.023 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writeback} duration: 942.9472754001617 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T09:12:24.023 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T09:12:24.102 INFO:teuthology.run:pass